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I. INTRODUCTION 


A. OVERVIEW 

Top management personnel often attempt to quantify or 
measure the value of information systems technology to their 
organization. Managers within the Information Systems (IS) 
community, in turn, debate and search for improved techniques 
to measure and, otherwise, make improvements to both existing 
and future systems. 

In 1984 and 1986, the Society for Information Management 
(SIM) along with the Management Information Systems Research 
Center (MISRC) conducted surveys to determine key information 
Systems issues facing IS executives/managers over the next 
five years. An issue ranked among the top ten in importance 
and one which has long been an IS management problem was 
system effectiveness and its measurement (Brancheau and 
Wetherbe, 1987). 

Inasmuch as the Department Of Defense (DoD) has a number 
of large, expensive information systems being used at various 
Medical Treatment Facilities, a simple and efficient method to 
measure and analyze the effectiveness of these systems is 
essential. This is a very difficult concept to narrow down 


because of differing opinions or ideas of value (ie; economic, 


efficiencv, effectiveness, user satisfaction, etc.). Many 
scholars of this problem have determined and agreed that all 
of these factors are important ones, however, there is a need 
to concentrate on the effectiveness of a system in terms of 
user interface. In other words, user satisfaction of the 
system. This is due to the fact that no matter how "good" the 
system may be, if the user is not satisfied with it, it 
becomes worthless. However, if the user is satisfied, the 
system may be considered to be effective, provided all other 


organizational objectives are met. 


B. RESEARCH OBJECTIVES 

This studv is a follow-on to the Evaluation of User 
Information Satisfaction of the Composite Health Care Svstem 
(CHCS) (Hurd, 1991) and will empirically evaluate the 
effectiveness of the Automated Quality of Care Evaluation 
Support System (AQCESS) and compare them to those found for 
CHES. A standardized, previously validated, satisfaction 
survey was administered to users of AQCESS at the Naval 
Hospital, Pensacola, Florida and Silas B. Hayes Hospital, Fort 
Ord, California to measure imbrinsic satisfaction 
characteristics. The results of the survev will be analvzed 
using relevant statistical methods in order to identifv and 
document problem areas if thev exist, as well as those areas 
considered to be positive aspects of the svstem. Upon 


completion, this studv will formulate a baseline measure of 


the AQCESS user's satisfaction at both sites and investigate 
possible cause and effect relationships. Future comparisons 
can be made using the same standard survey at the study 


hospital or at other facilities. 


II. THE AUTOMATED QUALITX OF CARE EVALUATION SUPPORT SXSTEM 


B. BACKGROUND 

The Tri-Service Medical Information Svstems (TRIMIS) 
Program Office was formallv created in Julv 1974, bv the 
Department of Defense Assistant Secretaries of Defense 
(Comptroller, Health and Environment). The program, which is 
now part of Defense Medical Systems Support Center (DMSSC) of 
the Office of the Assistant Secretary of Defense Health 
Affairs, was assigned the task of consolidating previous 
service efforts and to "improve the effectiveness and economy 
of health care delivery in the Army, Navy and Air Force." 
TRIMIS refined its mission objectives by stating that it would 
develop automated information systems for timely patient- 
centered health data. These systems would support medical 
services, clinical research, epidemiological studies and 
health care information. 

The Hospital Services Program Office (HSPO) developed a 
computer supported clinical records and patient administration 
system using the MUMPS language combined with certain 
utilities from the Veterans Administration File Manager. The 


system received extensive tri-service input and was designed 


incorporated comprehensive service-specific information to 
ensure accurate and reliable data accumulation. The svstem 
was also designed to be user-friendly (easy to learn, able to 
provide online assistance, and able to operate without 
dedicated computer operators/special environmental 
conditions). The MUMPS language allowed for Hardware 
flexibility (easily modified to correct problems and integrate 
additional requirements). Development of this system, 
however, was suspended following redirection of the TRIMIS 
program in March of 1984. (AQCESS User Manual, Issues 2.0, 


2:2, 2.4) 


B. AQCESS PURPOSE 

In August of 1984, the Health Affairs Office of the 
Assistant Secretary of Defense identified medical quality 
assurance as being a priority within the military health care 
system. The TRIMIS Office, along with representatives from 
the Army, Navy and Air Force, were requested by the 
Professional Affairs and Quality Assurance Office of the 
Deputy Assistant Secretary of Defense to develop and implement 
a computer supported information system at all DoD hospitals 
by 1985. The resulting system was the Automated Quality of 
Care Evaluation Support System (AQCESS). This system, also 
written in MUMPS, was designed to satisfy the following 
objectives: (1) Collect and report clinical, administrative 


and managerial information necessary to support inpatient 


administration of the medical qualitv assurance programs 
within DoD; (2) Improve the quality and timeliness of health 
care evaluation; and (3) Support the identification of 
variations that would adversely affect the quality of health 
care. (Assistant Secretary of Defense, Health Management 


Systems, 1984) 


C. AQCESS SYSTEM MODULES 
AQCESS is an interactive, mini-computer based, online 
computer system. It includes the following modules: 
1. Admission and Disposition 
Admission and Disposition addresses patient 
registration, admission, transfer, disposition, bed 
management, inpatient history and reporting. The user inputs 
patient information/data onto a formatted screen and has the 
capability to manipulate (ie; edit, view, cancel, track, 
print, validate) as necessary. From there, a number of other 
items may be generated (ie; register number, remarks section, 
inpatient history, query of patient database, general/specific 
reports). 
2. Clinical Records 
Clinical Records includes documentation on patient 
episodes, diagnosis and procedure data capture, patient day 
computation, record tracking and reporting. The user can 
collect, edit, validate, display, track various items, 


compute/maintain data and generate reports. 


3. Quality Assurance 
Quality Assurance supports inpatient and emergency 
room occurrence screening, problem tracking, offers solutions/ 
recommendations, provider profiling, credentialling, incident 
reporting, problem audit tracking, drug utilization, infection 
control, blood utilization review and reporting, risk 
management and surgical case review.  Additionallv, through 
monitoring and evaluation, the QA subsystem can gather data 
pertaining to the MTF's administrative procedures. It is then 
used to generate reports and trend analysis with respect to 
the quality of care. 
4. Ad Hoc Reporting 
Ad Hoc Reporting enables users to produce AQCESS 
reports in support of management and clinical research 
studies. It can produce very detailed reports, extract data 
from multiple files, compute new data from existing data, 
enable any data item to be ported to any deliverable form, use 
boolean logic selection, conduct multi-level sorting, perform 
computations, provide flexible report formatting, generate 
MUMPS code, support svstem securitv. 
5. Embosser Interface 
Embosser Interface supports a number of patient 
(input) information card embossers. The embosser produces a 


credit card-sized, raised letter card which is subsequently 


used much like a credit card (patient information obtained 
from card for input into the svstem). 
6. Interface to the Defense Enrollment Eligibilitv 

Reporting Svstem (DEERS) 

Interface to DEERS provides the capabilitv to check 
patient eligibilitv at the time of admission as well as input 
changes/corrections to both the AQCESS and DEERS databases. 

7. Business Office 

Business Office includes cashier processing and a 
number of other accounting and general business/office 
functions. It enables users to calculate inpatient/outpatient 
bills, review schedule of accounts, post accounts received, 
displav and update the rate schedule summarv and issue 
receipts/invoices. 

8. Outpatient Encounter 

Outpatient Encounter supports outpatient clinics and 
provides them with the capability to schedule, cancel or scan 
for appointments (patient appointments that are pending and to 
search for available slots), process a waiting list request, 
create/edit a schedule, check in patients for appointments and 
enter registration data for new patients. 

9. Emergency Room 

Emergency Room allows users to record patient 

emergency room visits; produces an automated SF Form 558 


(Emergency Treatment Record); produces the automated emergency 


room log; and stores, online, a historv of the encounter. The 
Emergencv Room module is no longer used because of reduced 
administrative manpower (system seems to be too time consuming 


under ER conditions). 


D. OPERATION 

AQCESS utilizes a centralized database and displays 
data/information bv means of a series of screens. It provides 
real-time information processing and generates several 
automated outputs on a daily and monthly basis. The terminal 
displays a sign-on screen on which the user enters a user ID 
and password. After the system validates this input, the 
AQCESS main menu is displayed. (If the user enters an invalid 
ID and/or pass-word more than a predetermined number of times, 
"User Entry" will lock the terminal and user ID.) The menu 
indicates the functions authorized to that user along with the 
specific functions which can be performed at that particular 
terminal. From the main menu, AQCESS data/information is 
subsequently displayed through a series of approximately 100 
screens associated with each sub-menu/ selection. The number 
of different responses users can make to each of the displays 
of data entered for each patient record (must consider a 
number of unique combinations) can be estimated as being 
infinite. There are a number of recovery and error correction 
procedures. However, software errors which cause the user to 


be deleted from the system are only corrected by the Systems 


corrected bv the Svstems Manager or technical assistants. If 
a user is authorized to access data via a svstems function, in 
most cases he/she is also authorized to update the data 
available through that function at anv time. An error will 
result when a user (e.g.,in admissions) attempts to update 
certain types of data elements (e.g.,the source of admission) 
because these data elements can only be updated/changed by the 
Systems/Corrections Manager. 

As patients are registered and admitted, the user inputs 
demographic data and the DEERS eligibility check is performed. 
An embossed card is issued, if requested, and it contains 
input information (register/account number, patient name, 
family member prefix, sponsor SSN, date of birth, sex) for 
access to patient's record. Upon patient's release from the 
MTF, further data is entered by the dispositions clerk, 
cashiering clerk and clinical records office. The database 
can then provide screen or hardcopy reports on current 
listings of patients admitted and discharged, current 
inpatients, patients on convalescent leave, patients 
subsisting elsewhere, patients on medical hold as well as 
listings of inpatient or delinquent bills. Once the patient 
has been admitted, the data is also tracked by the Quality 


Assurance Coordinator until he/she is discharged. 
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i. Svstem Management 
Svstem Management allows the Svstem Manager to modifv 
system tables (which define the valid entries for specific 
data fields), maintain hospital profile data, allows manager 
the flexibility to decide whether to assign register numbers 
automatically or manually, generate user ID's and passwords, 
assign functional privilege to users/terminals, broadcast 
system announcements, disable and re-enable user logon 
capability system-wide, control the active hours for the DEERS 
line, view the report queue and interrupt processing, archive 
records, perform system back-ups, list software error logs, 
load software updates and monitor disk space usage. 
2. Security 
Because of the sensitive nature of the data collected 
and reported by AQCESS, security of this data is of utmost 
importance. Briefly, system security is ensured by the 
following measures: 
e Use of systems functions and access to data displays and 
reports are restricted to authorized users and designated 
cerminals, as well as password. 


e Data is included on reports in the form of specific codes 
to protect confidentiality. 


e The system includes a time-out feature, causing the screen 
display to disappear if a terminal is left unattended past 
a predetermined time period. 


e A privacy act statement is printed as the banner/cover 


page of every system report. (AQCESS User Manual, Issues 
2 Oy 22 Agd) 
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E. SYSTEM BENEFITS 
Generally, the overall benefits of AQCESS include: 
1. automated data collection 
2. real-time processing 
. easily accessible clinical information 


3 
4. manipulation/analysis of data for easy generation of 
reports 


5. improved quality assurance monitoring 
6. improved security features 


7. improved system management capabilities 
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III. DESCRIPTION OF THE COMPOSITE HEALTH CARE SYSTEM 


A. BACKGROUND 

The Department of Defense (DoD) has pursued the goal of 
providing automated computer support to its hospitals and 
clinics since 1968. In Februarv 1979, the Mission Elements 
Need Statement (MENS), establishing the need for an automated 
computer svstem was approved. Under the direction of the Tri- 
Service Medical Information Svstem (TRIMIS) program office, 
stand-alone and integrated health care computer svstems were 


acquired, implemented and operated to support Pharmacy 


(TRIPHARM), Laboratorv (TRILAB), Radiologv (TRIRAD), 
Appointment and Scheduling (TRIPASS), Quality Assurance 
(AQCESS), and Hospital Information System (HIS). The 


knowledge and experience gained through the operation of these 
systems since 1979 was used to refine and validate the system 
requirements for a completely new and fully integrated 


information system the Composite Health Care System (CHCS). 


B. CHCS PURPOSE 

The primary purpose of CHCS is "to provide health care 
services that support military forces in fulfilling their 
required mission...." (Draft System Decision Paper, 1989). 


Improving the efficiency and effectiveness of military health 
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care delivery during peacetime and during mobilization through 
integration of information resources is the goal of CHCS. The 
major focus of CHCS is the integration of shared information 
resources to resolve the deficiencies of communication, 
decision support, and information processing. The Composite 
Health Care System is designed as a fully integrated medical 
information system that provides automated support of 
information requirements for military medical treatment 
facilities. CHCS supports the administrative functions and 
the delivery of health care with information retrieval 


services. 


C. DEPLOYMENT STATUS 

CHCS is undergoing the Operational Test and Evaluation 
(OT&E) phase and is currently deployed to Naval Hospital 
Charleston as one beta-test site. The role of a beta-test 
Site is to incrementally replace the TRIMIS systems with 
validated CHCS software. Continued validation of requirements 
and extensive evaluation of the CHCS software is conducted 


prior to the decision to deploy CHCS worldwide. 


D. OPERATIONAL FEATURES 

CHCS supports various functional areas in sharing 
information. Each functional area uses this information for 
its own purpose and communicates results/activities to other 


areas. Functional areas supported by CHCS include: 


14 


1. Patient Administration 
Patient administration does the registration of a 
patient into the system. Once in the computer system, the 
patient's demographic information is available for access by 
all other modules. The patient administration module also 
performs the functions to admit patients to the hospital, 
transfer patients between wards, and discharge patients from 
the hospital. The creation, update and closing of inpatient 
records is also performed by the patient administration 
module. 
2. Patient Appointment and Scheduling 
The Patient Appointment and Scheduling module provides 
for a centralized appointment service, a decentralized 
appointment service, or a combination of both depending on the 
desire of the hospital. The appointment service creates and 
maintains the appointment schedule for the hospital. A 
patient is given an appointment date and time to see the 
health care provider creating a centralized repository of 
information to produce operational reports of activity. 
3. Nursing 
CHCS supports inpatient nursing ina variety of areas. 
In the area of patient care, nursing uses the system for 
entering and obtaining the status of all physician orders, 
entering patient assessment data, and generating patient care 


plans. Documentation of patient progress is performed in the 
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automated nursing notes. Nursing unit management uses CHCS to 
generate shift care plans, patient management reports, drug 
administration time reports, and staffing requirement reports. 
Routine administrative functions are performed bv CHCS for 
nursing including: staff credentialling, documentation of 
continuing education and in-service training, and staff 
scheduling capabilities. 
4. Laboratory 

The laboratory uses the CHCS system for processing of 
orders for laboratory tests with automated reporting of test 
results. Specimens are processed, tracked, and reported 
through the system. All results are placed into the patients 
automated medical record and are immediately available for 
inquiry by health care providers. Automated control of blood 
bank operations including: blood acquisition, inventory and 
utilization are accomplished through CHCS. The laboratory is 
supported with ability to manage inventory control, register 
patients into the tumor registry, conduct a drug testing 
program, and generate a multitude of management reports. 

5. Pharmacy 

The pharmacy uses CHCS to process prescriptions for 
patients that are ordered by health care providers. The 
health care provider enters the prescription information at a 
terminal located in his office. The information is 


electronically transmitted to the pharmacy. The pharmacy can 
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immediatelv prepare the medication for dispensing before the 
patient leaves the phvsician's office. CHCS automaticallv 
performs checks for patient allergies, drug interactions with 
medications the patient is currently taking, drug dosage for 
the age of the patient, and records the prescription into the 
patient's medical record. In-patient order processing is 
accomplished in much the same fashion. On-line drug 
monographs are available for inquiry by medical staff at any 
terminal. Pharmacy management of inventory control and 
reporting are also accomplished. 
6. Radiology 

Orders for radiological procedures are electronically 
transmitted to the radiology department where the order is 
processed. Order tracking and results reporting are performed 
by the system.  Radiologv results are immediately available 
for inquiry by medical staff. Department management including 
inventory control is also conducted through CHCS. 

7. Clinical Dietetics 

The Clinical Dietetics service uses CHCS to obtain 
patient data needed to access the nutritional needs of the 
patient. This information is used to create patient diet 
plans, select patients of interest, patient menu selections 
and monitor patient nutritional data. CHCS is also used to 
maintain the nutrition clinic schedule and compute clinical 


dietetics workload data. 
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E. POTENTIAL BENEFITS 


In general terms, the overall benefits of the CHCS system 


include: 
1. reliable, timely, easily accessible Clinical 
information 
2. improved documentation for medicolegal issues 
3. improved communication 
4. improved patient satisfaction 
5. improved management efficiency 
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IV. USER INFORMATION SATISFACTION AND ITS MEASUREMENT 


A. INTRODUCTION 

while the cost of hardware and telecommunications 
continues to decrease, the cost of software development and 
maintenance for "systems" is continuously rising. 
Organizations spend considerable amounts of time and money to 
create and implement useful/successful information systems. 
The first 'hurdle' to be cleared is whether or not the system 
is really needed. If the need is there and the system is 
developed, the system's functionality must be evaluated (Is it 
doing what it's supposed to do?). Therefore, in addition to 
gauging the system by cost-benefit analysis, the effectiveness 


and intangible benefits of the system need to be evaluated. 


B. METHODS TO ASSESS BENEFITS/EFFECTIVENESS 

Several procedures have been suggested for assessing the 
benefits or effectiveness of an information system. Possibly 
due to the level of knowledge possessed by today's users, 
measuring user satisfaction is a common consideration in these 
procedures. User satisfaction is a very critical yardstick in 
measuring computer system success and failure. (Powers and 


Dickson, 1973) How can it be measured? 
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Svstem usage (Swanson, 1974; Conrath and Mignen, 1990) 
assumes that the effectiveness of an information svstem can be 
evaluated bv the amount of time the system is used. Simply, 
one uses an effective information system more than a less 
effective system. The assumption here is that the use of the 
system is optional to the user. If, however, the use of the 
system is mandatorv, the correlation between use and 
effectiveness can no longer be made. 

Others (e.g., Hamilton and Chervany, 1981) contend that 
effectiveness is determined by comparing performance to pre- 
determined objectives. TO assess the effectiveness of an 
information system, the task objective of the system is first 
agreed upon. Standards are developed to measure how well the 
information system performs each task. Some of the problems 
which emerge include: objectives and measures are often not 
sufficiently defined; easily implemented, efficiency-oriented 
measures are regularly used over complex, effectiveness- 
oriented measures; individual or conflicting interpretations 
of what the objectives and corresponding data measures are, 
often exist. 

Another frequently used procedure to determine the 
benefits/value of an information system is economic evaluation 
(cost-benefit analysis). The benefits of a system are weighed 
against the cost for development and operation of the system 
(Nolan, 1974). Although this appears objective and 


comprehensive on paper, in practice, cost-benefit analysis is 
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highlv subjective and difficult to conduct (Nolan, 1974). The 
difficulties in using this approach for research, as argued bv 
Ives et al. (1983), results from: (1) many costs and benefits 
are intangible and not easily recognized and/or converted into 
monetary equivalents; (2) unstructured, ad hoc decision making 
benefits are nearly impossible to objectively assess; (3) even 
when these items are determined by an organization, the data 
are generally unrecorded and not available for research (Ives 
et al., 1983). 

It has been theorized that evaluation of the effectiveness 
or success of an information system is best accomplished 
through the use of decision analysis. This is based upon 
studies which conclude that a user will interact with a system 
in order to obtain assistance in the decision making process 
if the system is perceived to be useful in that respect. In 
other words, there will be resulting productivity benefits. 
(Nolan, 1974) An effective information system supports a user 
by retrieving necessary information in the proper format, 
yields the required level of detail, displays it at the proper 
frequency and lends assistance to the decision-making process. 
However, the decision analysis approach suffers a significant 
disadvantage for use in scientific research in that it is non- 


quantifiable and cannot be easily replicated. 
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C. THE USER INFORMATION SATISFACTION CONSTRUCT 

The American College Encyclopedia Dictionarv defines 
satisfaction as 'the fulfillment of desires, expectations, 
needs or demands". Organizational Psychologist Bernard Bass 
(1965) generically defines satisfaction to mean ".... the 
extent to which the item is rewarding to us relative to how 
much better we might do elsewhere and what aspirations we 
have". 

A number of researchers have concluded that measuring User 
Information Satisfaction (UIS) is a viable substitute for 
measuring system success/effectiveness. Of note, however, are 
two empirical studies, (Aldag and Power, 1986; Gallupe and 
DeSanctis, 1988). These studies were conducted to find a 
relationship between satisfaction and actual system 
performance using objective third party judges. The results 
of these studies are conflicting and do not support the 
assumption that increased satisfaction indicates actual system 
performance increases. Although results have been mixed, 
researchers continue to diligently work towards validating and 
standardizing UIS measures. MIS practitioners and researchers 
agree that the user is the best gauge for determining the 
success or effectiveness of an information system. (Neumann 
and Segev, 1980; Ives et al., 1983; Bailey and Pearson, 1983; 
Baroudi and Orlikowski, 1988). 

The originators of the concept of UIS were Cvert and March 


(1963). Their research asserts that organizational behavior 
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continuallv imposes upon the manager the need for information. 
If a formal information svstem exits, the success or failure 
of that information svstem to meet the needs of the user 
either reinforces or frustrates the user's sense of 
satisfaction with that system. 

A user, who presumes that required information should be 
readily available through careful manipulation of an 
information system, carries out the necessary steps to 
retrieve that information. If the information is indeed 
readily available, then satisfaction with the system is 
reinforced. However, if the information is not readily 
available, the user must modify the search beyond the 
information system, possibly by some other less familiar 
procedure. As a result, the user becomes frustrated. 
Moreover, an elaborate exploration through an information 
system involves additional time and effort, hence, a decrease 
insproductivity. From the user's standpoint, information 
systems may actually hinder the user in carrying out the act 
of decision making (Nolan, 1965). 

Successive use by the user affords him/her the opportunity 
to continually evaluate the system. If over a period of time, 
the user discovers that the information system cannot retrieve 
specific information without the frequent use of ad hoc means, 
normally a response of user dissatisfaction with the 
information system occurs. The project is a failure if the 


end product does not satisfy those it was designed to serve 
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(Powers and Dickson, 1973). If, however, the user reliably 
obtains the required/ requested information through normal 
operation of the information system, satisfaction with the 
system will be reinforced and the information system is 


regarded as a success. 


D. THE MEASUREMENT OF USER INFORMATION SATISFACTION 

User Satisfaction, as a pervasive measure of svstem 
effectiveness, has gained wide acceptance as well as many 
improvements along the way. (Ives and Olson, 1979; Igbaria and 
Nachman, 1990). Neumann and Segev (1980) establish a 
correlation between a user's reaction to factors having to do 
with satisfaction and his/ her perception of their 
organization's performance. Swanson (1974) empirically found 
a high correlation between the user's appreciation of the 
system and his/her usage of its outputs. Powers and Dickson 
(1973) concluded that user satisfaction is the most critical 
factor for system success. Albeit there is no standard 
measure of satisfaction in these studies, it is maintained 
that user information satisfaction is an indicator of system 
usage and success. 

The level of user information satisfaction in the studies 
previously referred to is derived from a multitude of factors. 
Many of the users were asked to evaluate computer services in 
terms of their sense of satisfaction (Bailey and Pearson, 


1983). The factors measured in the various studies included: 
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accuracv, content, frequency, timeliness, reliability, 
assistance, adequacy, accommodation, communication, access, 
appreciation and flexibility. Each of these studies used a 
measure that was unique to that particular study (Ives et al., 
1983), nevertheless, taken as a whole, provide insight into a 
description of UIS. 

Validation of the UIS measures in these studies is 
limited, especially in handling threats to internal validity 
(Haga and Zviran, 1990). Bailey and Pearson's (1983) model 
and questionnaire for measuring and analyzing UIS seemed to 
escape this deficiency. It was based on previous studies and 
provides the basis of an instrument, later refined (Ives et 
al., 1983), and validated (Baroudi and Orlikowski, 1988) for 
use. Although the method employed to measure. user 
satisfaction by this tool is a pre-experimental design lacking 
pre-test/post-test or some type of control group (Campbell and 
Stanley, 1966), Conrath and Mignen (1990) assert that it 
represents a key contribution in the development of a standard 
instrument to measure user satisfaction. 

Bailey and Pearson (1983) described a model proposed by 


Lawler and Wanous (1972) for measuring user satisfaction: 


where: 
R,, = The reaction to factor j by the individual i 
The importance of factor j to individual i 


Wii 
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Using this model, the satisfaction of the user is measured 
as the weighted sum of the user's positive and negative 
reactions against a set of information svstem factors (Bailev 
and Pearson, 1983). The user's perception of a "generically 
good" information system is described as one composed of 
factors which have previously been deemed to be most important 
and a system with which the user is very satisfied. 

TO implement the model, a review of 22 studies dealing 
with computer-user interface was used to establish a set of 
factors representing the domain of user satisfaction. (Bailey 
and Pearson, 1983). The initial 36 factors generated from the 
literature review were expanded to 39 after further review by 
middle managers. They deduced that the 39 factors decided 
upon represent the domain of user satisfaction at a=0.01. 
Bailey and Pearson (1983) used four bipolar adjective pairs on 
a seven interval scale in order for users to explain and 
measure their perception of each factor. (the semantic 
differential technique; see Figure 4.1) The seven intervals, 
denoted by adverbial qualifiers, incrementally ranged from 
negative to positive feelings or vice versa. In addition to 
the ‘four adjective pairs, the user was required to test 
internal consistency and/or assign importance to each factor 
via two additional scales. 


Format of Output: The material design of the layout and 
display of the output contents. 


good : ; : 
simple : $ : 


readable : : : : : : 2 : unreadable 


useful : : : ; : : : : useless 
To me, this factor is 
important : : : : : : : > unimportant 


Figure 4.1 Illustration of questionnaire form 


Reliability coefficients of the Bailey and Pearson user 
information satisfaction questionnaire were determined to be 
very high, thus it was a reliable, valid instrument (Bailey 
and Pearson, 1983). It represented an important first step 
toward the development of a valid and useful UIS measure (Ives 
et al., 1983). 

Deese (1979) used the Pearson questionnaire at the Federal 
Computer Performance Evaluation and Simulation Center. He 
professed, "The results identified problems that would not 
otherwise have been discovered" and maintained that the user 
satisfaction questionnaire was a very useful and worthwhile 
tool (Deese, 1979). 

Ives, Olson and Baroudi (1983) chose to undertake an 
in-depth assessment of the Pearson (1983) questionnaire. By 
replicating the initial Bailey and Pearson study, they too, 
were able to reinforce the validity and reliability of the 
instrument. They presented several approaches to improve the 
quality of the original Pearson instrument. 

The goal of the suggested improvements was to establish a 
standardized "short form" instrument. Since Pearson found 


that the importance scale provided no additional information, 
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it was eliminated from the instrument. Scales which displaved 
undesirable psvchometric qualities were also eliminated. The 
number of items scaled within a question was reduced from four 
to two in order to reduce the time to complete the 
questionnaire. All of the scales in the original Pearson 
instrument were scored positively to the left and negatively 
to the right end of the scale. In order to reduce undesired 
outcome produced when a user simply marks down a column of 
responses, some of the scales were reverse scored, thereby 
increasing the reliability of the measure. The resulting 
short form was determined to substantially measure the concept 
of UIS that was originally proposed in the Pearson full 
instrument (Ives et al., 1983). 

The short form measure (Ives et al., 1983) was further 
subjected to testing by Baroudi and Orlikowski (1988) to 
examine its psychometric properties. The final short form 
questionnaire consisted of 13 questions with two items 
evaluated per question (see Appendix A). Each item is scored 
on a seven interval scale, ranging from -3 (dissatisfied) to 
+3 (satisfied) with zero indicating a neutral response. 

The total individual user information satisfaction score 
is calculated by averaging the responses of the 2 items for 


each question and summing the scores of the 13 questions. 


13 
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S, = total individual user satisfaction 
R, = response to the first item scale 
R, = response to the second item scale 


The range of total satisfaction can be from -39 to +39. 

Three factors: electronic data processing (EDP) staff and 
services (Factor A), information product (Factor B), and 
knowledge and involvement (Factor C) were found to comprise 
user satisfaction using factor analysis by Ives, Olson and 
Baroudi (1983). These subtotals are calculated as the average 
of the responses to questions loading into a particular 
factor. Questions 1, 2, 6, 11, and 12 load heavily into the 
electronic data processing (EDP) staff and services factor. 
Questions 7, 8, 9, 10 and 13 load heavily into the information 
product factor. Questions 2, 3, 4, and 5 load heavily into 
the knowledge and involvement factor. The factor subtotals 
will range from -3 to +3 in value. Averages are used for 
meaningful comparison between the three factor scores. 

Data gathered for the study came from 358 employees, 
mostly clerical and support personnel, of 26 New York area 
organizations. Construct validity was determined first by 
examining the relationship between each scale and the total 
UIS score. Factor analysis using varimax rotation converged 
to a three factor solution accounting for 68% of the variation 
in 5 iterations. This provides strong evidence of construct 


validity (Baroudi and Orlikowski, 1988). 
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Convergent validitv was substantiated bv comparing an 
interview measure of satisfaction with the instrument measure 
in two groups. One group of users were generally pleased and 
satisfied with the information system and the other group of 
users were generally dissatisfied. Administration of the 
instrument indicated a statistically significant difference by 
t-test in the groups at p < .001 (Baroudi and Orlikowski, 
1988). 

The total satisfaction and subtotal scores reported a 
reliability level above the .80 required for research. This 
confirms the instrument to be internally consistent and 
reasonably free from measurement error. 

The work by Baroudi and Orlikowski (1988) and resulting 
short-form UIS instrument provides a reliable and valid 
measure of user information satisfaction. It furnishes a 
meaningful, standardized measure of overall satisfaction with 
an information system as well as specific information about 
satisfaction within the sub-factors of electronic data 
processing (EDP) staff and services, information product, and 


user knowledge and involvement. 
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V. RESEARCH METHODOLOGX 


A. INTRODUCTION 

The purpose of this research is to identify the 
characteristics of user information satisfaction (UIS) for 
users of the AQCESS system and compare the findings to those 
obtained for CHCS (Hurd, 1991). The study is based on 
statistical analvsis of empirical data collected from AQCESS 
users at the Naval Hospital, Pensacola, FL and Silas B. Havs 
Hospital, Fort Ord, CA. The following sections will discuss 
the survey instrument, data collection methodology and methods 


of statistical analysis utilized in the study. 


B. THE SURVEY INSTRUMENT 

Avoiding the obstacles associated with developing a 
comprehensive survey questionnaire, the previously developed, 
psychometrically evaluated and validated short-form 
questionnaire of Baroudi and Orlikowski (1988) was used 
without alteration. A copy of the complete survey 
questionnaire is included in Appendix A. The questionnaire is 
composed of two sections: a demographical section and a user 


information satisfaction section. 
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1. Demographic Section 

The first section of the questionnaire requested 
general information. Regarding questions 1-5, the respondents 
were asked to give their hospital division/department, job 
description, highest level of education, age and gender, 
respectively. To determine the user's experience level with 
the system, the sixth question asked for the length of time, 
in months, the respondent had used AQCESS. Further 


determination of the user's experience was sought in questions 


seven and eight. Question seven asked if the user had 
previously used other computer systems. If the respondent 
answered "yes" to question seven, he/she was asked if the 


previous system was a health care/hospital information system. 
2. User Information Satisfaction Section 

The second part of the survey directly addressed the 

issues of user information satisfaction. The instrument was 

designed to obtain the user's opinion on how well the AQCESS 

system functioned. It consisted of thirteen questions; each 


question having two bipolar, adjective scales for responses. 


ti tt 


Each item was to marked with an 'x' in one of seven values, 
ranging from -3 (extremelv dissatisfied) to 43 (extremelv 
satisfied), with zero indicating a neutral response. 

The questions asked can be categorized into one of 


three factors of user satisfaction. 
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Management information Department/Information 


Management Division (MID/IMD) Staff: This factor is the 
respondent's perception of the attitude and responsiveness of 
the MID/IMD staff as well as their relationship with said 
staff. The MID/IMD staff provides only local support and act 
as a liaison to the Defense Medical Systems Support Center 
(DMSSC) which manages and administers the AQCESS program. 
Thus, this factor also takes into account services provided by 
BMSSC. 

Information Product: This factor is the respondent's 
perception of the quality of output delivered by AQCESS. 

Knowledge and Involvement: This factor is the 
respondent's perception of the quality of training provided, 
their understanding of the system, and their participation in 


its development/modifications. 


C. SAMPLE AND DATA COLLECTION 

Naval Hospital, Pensacola and Silas B. Hays Hospital, Fort 
Ord are two of a multitude of U.S. Navy, Army, Air Force and 
Coast Guard sites which operate AQCESS. The sample population 
then consisted of personnel from in/out-patient, 
clinical/technical and administration areas. 

Local points of contact acted as distribution and 
collection agents for the surveys. The questionnaires, each 
accompanied with a cover letter (Appendix B), were delivered 


to a point of contact at each site. The points of contact 
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distributed 500 questionnaires at Naval Hospital, Pensacola 
and 200 at Silas B. Havs Hospital, Fort Ord (with return 
envelopes and instructions) to all department personnel within 
the hospital operating the AQCESS modules. The survevs were 
completed and returned bv the respondents to the points of 
contact in sealed envelopes and were then forwarded to the 
researchers for analvsis. 

There were 214 questionnaires returned from Pensacola and 
71 from Fort Ord, of which 174 and 58 survevs, respectivelv, 
had complete information. This represents a final return rate 


of 34.8% and 29%, respectively. 


D. STATISTICAL ANALYSIS METHODS 
Data from completed surveys were coded and entered into a 
spreadsheet program. The spreadsheet program was used to 
perform the statistical analysis. The specific procedures 
used in the analysis will be examined next. 
1. Respondent’s Satisfaction Index 
Each respondent's total satisfaction index was 


calculated for the 13 questions using the following formula: 


13 
S= y RaijtRpij 
a de 
7-1 2 
where: 
Si = Satisfaction Index for respondent i 


Ra; = RESPONSE TOPES SUS eo questione eroe 
respondent 1 
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Rm; = Response to second item scale of question j for 
respondent i 


2. Total Satisfaction Index 
The overall or total satisfaction index for the survey 
group is calculated by averaging the respondent's satisfaction 
index to find the mean. 
3. Question Averages 
The mean response to each of the thirteen questions is 
calculated for comparison of individual questions. 
4. Factor Subtotals 
The thirteen questions can be grouped into three 
factor subtotals, as defined by Ives, Olson, and Baroudi 
(1983): MID/IMD staff and services (Factor A); Information 
product (Factor B); and Knowledge and involvement (Factor C). 
The mean of the individual questions' averages is calculated 
to find the factor subtotals. Questions 1, 2, 6, 11, and 12 
are used for the MID/IMD staff and services subtotal. 
Questions 7, 8, 9, 10, and 13 are used for the information 
product subtotal. Questions 2, 3, 4, and 5 are used for the 
knowledge and involvement subtotal. 
5. Comparison of Groups 
The respondents were categorized by the type of use of 
the AQCESS system. Three work groups were identified: (1) 
Physicians; (2) Ancillary; and (3) Administration. The 
individual questions' averages, total satisfaction indexes, 


and sub-factor totals were compared for one group verses the 
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combination of the other two groups using a t-test with the 
significance level of alpha = 0.05. The t-test is used to 
determine if there is a significant difference between the 


arithmetic mean value of two groups. 
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VI. DESCRIPTIVE FINDINGS (NAVAL HOSPITAL, PENSACOLA) 


A. DEMOGRAPHIC FINDINGS 

Of the 174 respondents, 76 were male and 98 were female. 
The age of the respondents ranged from 19 to 77 with a mean of 
36 years old. 

1. Work Center Distribution 

The hospital department work centers reported were 

in/out-patient areas including: administration (ie; 
Appointment and Scheduling, Medical Service Accounting, 
Emergency Room and Quality Improvement), clinics (ie; Physical 
Therapy, 0B/GYN, Pediatrics, etc.), laboratory (ie; Radiology, 
Pathology, etc.) and other departments. Figure 6.1 reflects 


the distribution by work center. 


PENGĦCOLM NA ATTA 
WORKCENTERS 


OTHER (2) 


CLINIC (104) 





Figure 6.1 
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2. Job Description Distribution 

The job descriptions reported are from a multitude of 

specific functions but can be categorized as: 

1. Technician - a hospital corpsman functioning in a 
medical-technical capacity (pharmacy technician, 
laboratory technician, clinic corpsman) 

2. Physician - a medical doctor 

3. Health Professional = a licensed medical 


professional other than a physician (nurse, 


pharmacist, physical therapist, bio-medical 
officer) 
4. Other - a person not listed above (hospital 


administration non-medical clinic staff) 
Figure 6.2 reflects the sample distribution by job 


description. 


PENSACOLA NAVAL HOSPITAL 
JOB DESCRIPTION 


OTHER (85) 





Figure 6.2 
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k Group Distribution 


the purposes of this investigation, the study 


population was segregated according to the assumed primary use 


of the computer system. The categories provide a clear 


separation of work groups from an organizational and cultural 


standpoint. 


study. 


Figure 


E 


These groups are used extensively throughout the 


Administrative - use of the system for indirect 
medical purposes (patient registration, 
appointment scheduling and administrative 


reporting) 


2. Ancillary - use of the system for direct medical 


support (prescription filling, laboratory specimen 


processing) 


3. Physician - use of the system by physicians 


bid 


(accessing patient historical data, performance 
reports) 


represents the distribution of respondents 


according to their main use of the system. 


4. Education Distribution 
The level of education reported by the respondents 
represents a well educated user population. A little more 


than 79 percent of the users have had at least some college 


education with nearly 31 percent having completed a Bachelor, 
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Master, Medical or PhD degree. Figure 6.4 represents the 


breakdown bv education. 
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Figure 6.3 
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Figure 6.4 
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B. COMPUTER SYSTEM USE 

The length of time the respondents reported as having used 
the AQCESS computer system ranged from one to 72 months with 
a mean Of 21 months. Of the 174 respondents, 63 had used a 
computer system previously. 58 of the 63 users had used a 
health care information system before using AQCESS. Although 
many of the respondents had experience with computers, this 
was the first exposure to a health care information system for 


67 percent of the users. 


C. SATISFACTION FINDINGS 

The survey results for user satisfaction are compared for 
the three work groups in three areas: 1) overall satisfaction, 
2) responses to individual questions, 3) three group factors: 
MID staff and services; information product; knowledge and 
involvement. The actual values of responses to individual 
questions reported in the survey may be significant in 
themselves. However, the relative scores are useful for 
comparison among various work groups, organizations and 
information systems. (Baroudi and Orlikowski, 1988) The 
administrative group composes nearly 48 percent of the survey 
population, the ancillary group composes 40 percent and the 


remainder consists of physicians. 
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1. Overall Satisfaction (Pensacola) 

The overall satisfaction was determined by the mean of 
the sum of the 174 survey responses to the thirteen survey 
questions from the work groups at Pensacola. The mean overall 
satisfaction index of the combined work groups was 11.16 on a 
scale ranging from -39 to 439. Figure 6.5 illustrates a 
comparison of the overall satisfaction index of the work 
groups at Pensacola. The survev shows that the administrative 
group displavs the highest overall satisfaction (13.81), 
closelv followed by ancillary (13.73), with physicians 
registering the least overall satisfaction (-4.16). Using a 
t-test, the level of satisfaction for physicians was 
statistically significant lower at alpha = 0.05 than remaining 


groups. 
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Figure 6.5 
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2. Individual Questions 
The 13 questions on the survey requires two responses 
from the respondent. A number of the response scales are 
reverse scored to nullify the effects of a respondent who 
simply marks down one column of the questionnaire (Baroudi and 
Orlikowski, 1988). Figure 6.6 displays the average level of 


satisfaction for each question, by work group, for Pensacola. 
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It is easily discernable that some questions exhibit a 
relatively high or low level of satisfaction when compared to 
the other questions. Questions 1, 6 and 11 indicate a high 
level of user satisfaction. These questions deal with issues 
concerning the Management Information Department (MID) staff. 
However, questions 2, 3 and 12 stand out as exhibiting a 


relatively low satisfaction index. Questions 2 and 12 are 
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concerned with software development/modification and question 
37e training- The following discussion will examine each 
question by work group. 

Figure 6.7 illustrates Pensacola physician's responses to 
individual questions as they compare to the remainder of the 
survey group. Figure 6.8 represents the ancillary group's 
responses as compared to the remainder of the survey groups. 
Figure 6.9 is the administrative group's responses as compared 


to the remainder of the survey groups. 
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Figure 6.7 


a. Question 1 - Relationship with the Management 
Information Department (MID) Staff 
This question measures the relationship between 


users and the Management Information Department. The level of 
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satisfaction towards the MID staff is relativelv high. It has 
the highest satisfaction index. The administrative group 
showed the highest level of satisfaction of the three groups. 
The ancillarv group was. slightiv lower. The phvsicians 
revealed a statistically significant lower difference at alpha 


270.05: 
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Figure 6.8 
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Figure 6.9 
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b. Question 2 - Processing of Requests for Changes 
to Existing Systems 
This question is concerned with the length of time 
required and/or timeliness of changes to AQCESS. This was the 
lowest scoring question which indicates a user perceived 
problem. Physicians were the least satisfied with a 
statisticallv significant lower difference at alpha = 0.05. 
The administrative users displayed a statistically significant 
higher difference at alpha = 0.05. 
c. Question 3 - Degree of Training Provided to Users 
The response to this question indicates the user's 
perceived level of training to use the system. The level of 
satisfaction of the administrative and ancillary groups was 
nearly equal. The physicians, however, showed a statistically 
significant lower difference at alpha = 0.05. 
d. Question 4 - User's Understanding of the System 
The response to this question is the respondent's 
self-reported assessment of his/her ability to interact with 
the system. Physicians were statistically significant lower 
at alpha = 0.05. Whereas, the administrative group showed a 
statistically significant higher difference at alpha = 0.05. 
The ancillary group indicated they were substantially less 
Satisfied with the amount of training they received (question 


2) as compared to their understanding of the system. 
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e. Question 5 - User's Feeling of Participation 
The responses to this question yield a measurement 
of the user's perception of participation in the system. The 
administrative group had a statistically significant higher 
perception of participation than the other groups. 
Converselv, phvsicians had a statisticallv significant lower 
difference at alpha = 0.05. The ancillary group's feeling of 
participation was slightly lower than the administrative, but 
Nee statistically significant. 
f. Question 6 - Attitude of the Management 
Information Department (MID) Staff 
The intent of this question is to gauge the user's 
perception of the attitude of the MID staff. In particular, 
the staff's attitude towards rendering services. The response 
to this question was positive for all three groups. The 
administrative group was statistically Significant higher 
while the physicians were statistically significant lower at 
alpha = 0.05. However, this question ranked third among the 
physician's positive responses. 
g- Question 7 - Reliability of Output Information 
The user's response to this question indicates 
his/her view of consistency and dependability of the output 
information. Administrative and ancillary groups responses 
were positive and approximately the same. Phvsicians' 


response was statistically significant lower at alpha = 0.05. 


47 


h. Question 8 - Relevancy of Output Information (to 
intended function) 

This question measures the degree of compatibility 
between what the user wants or requires and what is provided 
by the information products and services. Administrative 
group showed a statistically significant higher difference at 
alpha = 0.05. The physicians, however, displayed a 
statistically significant lower difference at alpha = 0.05. 
The ancillary group was slightly lower than the administrative 
group with no statistical significance. 

i. Question 9 - Accuracy of Output Information 

The user's opinion of correctness of the output 
information is captured by this question. Administrative 
group was statistically significant higher while physicians 
were statistically significant lower. Ancillary was not 
statistically significant, but was equally satisfied as the 
administrative group. 

j. Question 10 - Precision of Output Information 

Precision of output information is the variability 
of output information from that which it purports to measure. 
The administrative and ancillary groups were not statistically 
significant, but the administrative group's response was 
approximately twice that of the ancillarv group's. The 
physicians had a statistically significant lower difference at 


alpha = 0.05. 
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k. Question 11 - Communication with the Management 
Information Department (MID) Staff 
The response to this question is a basis for 
determining user perception of the manner and methods of 
information exchange between users and the MID staff. The 
results for the administrative and ancillary groups were not 
statistically significant, but instead, the scores were nearly 
even. Physicians did, however, show a statistically 
significant lower difference at alpha = 0.05. 
1. Question 12 - Time Required for New Systems 
Development 
This question is designed to assess the user's 
perception of the elapsed time between his/her requests for 
new applications and the design, development and the 
implementation of the application systems by the contractor or 
outside service representative. The administrative and 
ancillary groups were nearly neutral. The physicians, however 
displayed a statistically significant lower difference at 
alpha = 0.05. 
m. Question 13 - Completeness of Output 
The response to this question was intended to help 
measure the user's impression of the comprehensiveness of the 
content of the output information. The administrative group 


were statistically significant difference higher, while the 
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physicians were statistically significantly lower at alpha = 
0205: 
3. Grouped Factors 

The original user's information satisfaction survey 
instrument sub-divided the thirteen questions into three 
grouped factors. These factors are designated as EDP 
(MID/IMD) staff and services, information system product, and 
knowledge and involvement. (Ives, Olson and Baroudi,1988) The 
three factors are calculated from the averages of the 
component questions. MID staff and services (Factor A) is 
sub-divided into two separate components: local MID staff 
functions (MID) and contractor or Defense Medical Systems 
Support Center (DMSSC) functions (Con). Factor A (MID) is 
derived from questions 1, 6,and 11; Factor A (Con) from 
questions 2 and 12; information system product (Factor B) from 
questions 7, 8, 9, 10 and 13; and knowledge and involvement 
(Factor C) from questions 2, 3, 4, and 5 (Hurd,1991). 

The administrative group's and phvsicians' level of 
satisfaction were statisticallv different at significance 
level alpha = 0.05. The administrative group was the most 
satisfied and the physicians were the least satisfied of all 
the groups. Figure 6.10 represents the work groups' level of 
satisfaction with relation to sub-total factors. Figure 6.11 
represents factor satisfaction of physicians compared to the 


other work groups. 
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Figure 6.12 represents factor satisfaction of 


ancillarv personnel compared to the other work groups. 
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Figure 6.12 


Figure 6.13 represents factor satisfaction of 


administrative personnel compared to the other work groups. 
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Figure 6.13 
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Figure 6.14 represents the group factor satisfaction 
indices of administrative personnel compared to ancillarv 


personnel. 
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Figure 6.14 


a. Factor A (Local MID Staff and Services) 

This factor is the respondent's self-reported 
assessment of the attitude and responsiveness of the local MID 
staff and the services thev provide, as well as, the qualitv 
of the relationship with the MID personnel (Baroudi and 
Orlikowski). As seen in Figure 6.10, the administrative 
personnel displaved the highest level of satisfaction with a 
statistically significant difference at alpha = 0.05. The 
lowest level of satisfaction was displayed by the physicians, 


however, it is exceptionally high in comparison with their 
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responses to the other factors. The phvsicians were 
statistically significant at alpha = 0.05. 
b. Factor A (Contractor, DMSSC) 

This factor represents the responsiveness and 
services of outside software programmers and management. It 
received the lowest level of satisfaction of any factor. All 
work groups rated this factor extremely low, with 
administrative personnel recording the highest level of 
satisfaction and the phvsicians scoring the lowest level of 
satisfaction. The administrative group showed a statisticallv 
significant higher difference while the phvsicians showed a 
statistically significant lower difference at alpha = 0.05. 

c. Factor B - Information Product 

This factor represents the users perception of the 
quality of output delivered by the information system (Baroudi 
and Orlikowski, 1988). The administrative group scored the 
highest level of satisfaction with a statistically significant 
difference at alpha = 0.05. The physicians recorded the 
lowest level of satisfaction with a statistically significant 
difference at alpha = 0.05. 

d. Factor C - Knowledge and Involvement 

This factor represents the respondents' self- 
reported assessment of the quality of training provided, their 
understanding of the system, and their participation in its 


development (Baroudi and Orlikowski, 1988). There was a 
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statisticallv significant difference between the level of 
satisfaction of the administrative group and the phvsicians at 
alpha = 0.05. The administrative group and ancillary group 
rated the satisfaction level nearly the same. The physicians 
scored this factor the lowest of all the group factors. 

4. Time of System Use 

Igbaria and Nachman (1990) found that users with 
better computer background are likely to be more confident in 
their ability to use the information system and more satisfied 
with experience. Generally, users become more skilled with an 
information system over time. Therefore, the assumption is 
that the respondents who use the system the longest will be 
more experienced and record a greater level of satisfaction. 

The respondents were separated into subgroups of users 
with less than six months, six months to ll months, and 
greater than 11 months experience. Almost sixty percent of 
the respondents had 11 months or greater experience with the 
AOCESS system. Figure 6.15 represents the breakdown of 
respondents by time of system use. 

The respondents who used the system more than or equal 
to 11 months generated the lowest level of user satisfaction. 
Conversely, the respondents with 6 to 11 months of system use 
recorded the highest level of satisfaction with the system. 


Regression analysis indicated no correlation (r < 0.02) 
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between the time of svstem use and level of user satisfaction 


with the svstem for anv factor. 
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Figure 6.16 shows the level of satisfaction with AQCESS 


verses time of svstem use. 
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Figure 6.16 
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Breaking the work groups down bv time of svstem use, 
there was still no correlation between time of svstem use and 
level of satisfaction of anv factor bv regression analvsis. 
The following discussion will describe the level of user 
satisfaction for the work groups over time. 

The phvsicians with less than six months svstem use 
had a satisfaction index of 2.625. Unlike the ancillary and 
administrative groups, the physicians' level of satisfaction 
decreased to -3.25 between six to 11 months experience. The 
level of satisfaction for the physicians with more than 11 
months experience show a decrease at -6.36, a downward trend 
consistent with the other work groups. Figure 6.17 
illustrates the physician's level of satisfaction with respect 


to time of system use. 
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The ancillarv group's level of satisfaction displavs 
a similar trend to the administrative group. The satisfaction 
index for users with less than six months experience was 
slightly lower (18.17) than users with six to ll months 
experience whose level of satisfaction rose to 19.8. 
Surprisingly, the level of satisfaction for users with more 
than 11 months of experience drops dramatically to 9.4. 
Figure 6.18 illustrates the ancillary group's level of 


satisfaction with respect to time of system use. 
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Figure 6.18 


Although the trends for both the administrative and 
the ancillary group's level of satisfaction are nearly the 
same, the administrative group's level of satisfaction amongst 
the user subgroups is more pronounced. The satisfaction index 


for users with less than six months experience was lower at 
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15.6 than users with six to 11 months experience whose level 
of satisfaction rises steeply to 22.35. The level of 
satisfaction for users with more than 11 months of experience 
drops sharply to 9.4. Figure 6.19 illustrates the 
administrative group's level of satisfaction with respect to 


time of svstem use. 
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Figure 6.19 


VII. DESCRIPTIVE FINDINGS (SILAS B. HAYS HOSPITAL, FORT ORD) 


A. DEMOGRAPHIC FINDINGS 

Of the 57 respondents, 10 were male and 47 were female. 
The age of the respondents ranged from 19 to 65 with a mean of 
34 years old. 

1. Work Center Distribution 

The hospital department work centers reported were 

in/out-patient areas including: other (ie; Patient 
Administration Division, Quality Assurance, etc.) and clinics 
(ie; OB/GYN, Pediatrics, etc.). Figure 7.1 reflects the 


distribution by work center. 
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Figure 7.1 
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2. Job Description Distribution 
The job descriptions reported are from a multitude of 
specific functions but can be categorized as: 

1. Technician - a hospital specialist functioning in 
a medical-technical capacity (laboratory 
technician, clinical technician, medic) 

2. Other - a person not listed above (hospital 
administration non-medical clinic staff) 


Figure 7.2 reflects the distribution by job description. 


SIMS ao Rio eii 
JOB DESCRIPTION 


TECHNICIAN (2) 


DEE 





Ss 


OTHER (55) 





Figure 7.2 


3. Work Group Distribution 
For the purposes of this investigation, the studv 
population was segregated according to the assumed primarv use 


of the computer svstem. The categories provide a clear 
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separation of work groups from an organizational standpoint. 


These groups are used extensivelv throughout the studv of the 


Silas B. Havs Hospital sampie (Fort Ord). 


il 


2. 


Administrative - use of the system for indirect 
medical purposes (patient registration, 
appointment scheduling and administrative 


reporting) 
Ancillary - use of the system for direct medical 
support (prescription filling, laboratory specimen 


processing) 


Figure 7.3 represents the population of users/respondents 


according to their main use of the system. 
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4. Education Distribution 
The level of education reported by the users 
represents a fairly well educated user population. 
Approximately 67 percent of the users have at least had some 
college education. 12 percent completed a Bachelors or 
Masters degree with 5 percent having worked on a Masters 


degree. Figure 7.4 represents the breakdown by education. 
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B. COMPUTER SYSTEM USE 

The length of time users reported as having used the 
AQCESS computer svstem ranged from one month to 72 months with 
a mean of 19.4 months. Of the 57 respondents, 22 had used a 
computer system previously. 16 of the 57 users had used a 


health care information system before using AQCESS. Although 
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many of the respondents had experience with computers, this 
was the first exposure to a health care information system for 


72 percent of the users. 


€. SATISFACTION FINDINGS 

The survev results for user satisfaction are compared for 
the two work groups in three areas: 1) overall satisfaction, 
2) responses to individual questions, 3) three group factors: 
Information Management Division (IMD) staff and services; 
information svstem product; knowledge and involvement. The 
actual values of responses to individual questions reported in 
the survev mav be significant in themselves. However, the 
relative scores are useful for comparison among various work 
groups, organizations and information svstems. (Baroudi and 
Orlikowski, 1988) The administrative group composes nearlv 97 
percent of the survev population and the remainder consists of 
the ancillarv group. 

1. Overall Satisfaction (Fort Ord) 

Overall satisfaction is measured by the mean of the 
sum of the responses to the survey questions. The mean survey 
score index was 10.47 on a scale ranging from -39 to +39. 
Figure 7.5 presents a comparison of the overall satisfaction 
index of both groups and the overall mean satisfaction. The 
Survey shows that the ancillary group displays the highest 
overall satisfaction index (13.0), with the administrative 


group recording the lower overall satisfaction (10.38). Using 
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a t-test, there was no significant difference between the 
ancillary group and the administrative group at alpha = 0.05. 
2. Individual Questions 

The 13 questions on the survey require two responses 
each from the respondent. A number of the response scales are 
reverse scored to nullify the effects of a respondent who 
simply marks down one column of the questionnaire (Baroudi and 
Orlikowski, 1988). Figure 7.6 shows the average level of 
satisfaction for individual questions by work group. 

Similar to the results discussed in chapter six, 
questions 1, 6 and 11 indicate a high level of user 
satisfaction. These questions deal with issues concerning the 
IMD staff. However, questions 2, 3 and 12 stand out as 
exhibiting a relatively low satisfaction index. Questions 2 
and 12 are concerned with DMSSC services and modifications and 
question 3, training. The following discussion will examine 
each question by work group. 

a. Question 1 - Relationship with the Information 

Management Division (IMD) Staff 
As noted earlier, the level of satisfaction towards 
the IMD staff is relatively high. Although not statistically 
significant at alpha = 0.05, the ancillary group showed the 


higher level of satisfaction between the two groups. 
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Figure 7.5 
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Figure 7.6 


66 


b. Question 2 - Processing of Requests for Changes 
to Existing Systems 
This question received the lowest satisfaction 
index rating. The response was neutral for the ancillary 
group and negative for the administrative group. A t-test 
indicated no statistically significant differences at alpha = 
0.05: 
C. Question 3 - Degree of Training Provided to Users 
The level of satisfaction for the ancillary group 
was higher than that of the administrative group but not 
statistically significant difference higher at alpha = 0.05. 
d. Question 4 - User's Understanding of the System 
There was no statistically significant difference 
between the administrative and ancillary groups at alpha = 
0.09. Again, the ancillary group displayed a higher 
satisfaction index. 
e. Question 5 - User's Feeling of Participation 
In response to this question, the administrative 
and the ancillary groups responded positively with no 
statistically significant difference at alpha = 0.05. 
f. Question 6 - Attitude of the Information 
Management Division (IMD) Staff 
The response to this question (the second highest 
rated question) was favorable with no statistically 


significant difference at alpha = 0.05. 
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g. Question 7 - Reliability of Output Information 
The responses were similar with no statistically 
significant differences between the groups. 
h. Question 8 - Relevancy of Output Information (to 
intended function) 
Both groups responded comparabiv without 
statisticallv significant differences. 
i. Question 9 - Accuracy of Output Information 
The ancillary group was neutral regarding this 
item. The administrative group was slightly more positive, 
but there were no statistically significant differences at 
alpha = 0.05. 
j. Question 10 - Precision of Output Information 
The response by both groups was somewhat favorable 
with the administrative group's satisfaction index being a 
little higher. There were, however, no statistically 
significant differences. 
k. Question 11 - Communication with the Information 
Management Division (IMD) Staff 
All questions concerning the IMD staff received 
high satisfaction levels from both groups (highest rated 
question response). Neither of the groups' responses were 


statistically different by t-test at alpha = 0.05. 
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1. Question 12 - Time Required for New Systems 
Development 
The groups were nearly neutral regarding this 
issue. There were no statistically significant differences at 
alpha = 0.05. 
m. Question 13 - Completeness of Output 
Both groups responded similarly with no 
statistically significant differences at alpha = 0.05. 
3. Grouped Factors 
As discussed in chapter VI, the questions are grouped 
into three factors that comprise user satisfaction. These 
factors: IMD staff and services, information system product, 
and knowledge and involvement, are calculated through averages 
of the component questions. IMD staff and services (Factor A) 
1s sub-divided into two separate components: local IMD staff 
functions (IMD) and contractor/DMSSC functions (CON). 
Neither of the work groups’ satisfaction level for any 
of the factors were statistically different at significance 
level alpha = 0.05. Figure 7.7 represents the ancillary and 
administrative groups’ level of satisfaction in relation to 
the group factors. 
a. Factor A (Local MID Staff and Services) 
In Figure 7.7, the ancillary group displayed the 


highest level of satisfaction but it was not statisticall; 
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significant different from the administrative group at alpha 
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b. Factor A (Contractor Services) 
This factor received the highest level of 
satisfaction of anv factor. The ancillarv group was not 


statisticallv significant different from the administrative 
group at alpha = 0.05. 
C. Factor B - Information Product 
This factor is the second highest rated of the four 
and there is no statistically significant difference between 


the ancillary and administrative groups at alpha = 0.05. 
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d. Factor C — Knowledge and Involvement 
This factor showed no statisticallv significant 
difference between the level of satisfaction for the ancillarv 
or administrative groups. 
4. Time of Svstem Use 

As stated in chapter VI, it is expected that the more 
experienced users will exhibit a greater level of 
satisfaction. 

Figure 7.8 displavs the separation of users into 
subgroups having less than six months experience, six to ll 


months, and greater than 11 months. 
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Figure 7.8 


The ancillary group had no users with less than six 


months nor greater than 11 months of system use. Therefore 
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there is reallv no trend to report for this group.  However, 
it is worthy to note that the two respondents displayed a high 
level of satisfaction with an index of 13.0. Figure 7.9 
depicts the ancillary users' satisfaction index versus the 


time of system use. 
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Figure 7.9 


The administrative group's level of satisfaction tends 
to increase with experience and time of use. Respondents, 
with less than six months use, recorded a satisfaction index 
of 5.3. As the users gained experience (six to 11 months), 
the level of satisfaction increased to 10.8. For users with 
greater than 11 months experience, the satisfaction index 
elevated to 12.06. Figure 7.10 represents the administrative 


users' satisfaction index versus the time of system use. 
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Figure 7.10 
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VIII. @COMPARISON OF RESULTS FROM FORT ORD AND PENSACOLA 


A. INTRODUCTION 

when the user satisfaction survey questionnaire is 
assessed within a single organization, the UIS scores are 
indicative of the general level of user satisfaction with a 
specific information system. The scores, positive or 
negative, are on its own an important finding. However, the 
scores are useful for comparisons across different users of 
the same information system (to pinpoint the problems 
particular users may be experiencing), as well as for 
comparisons across various information svstems (to highlight 
specific information systems that may be problematic). 
(Baroudi and Orlikowski, 1988). Silas B. Hays Army Hospital 
(Fort Ord) and Pensacola Naval Hospital are two different 
organizations with similar information systems (AQCESS). This 
chapter will compare the user information satisfaction (UIS) 
measurements recorded from Fort Ord with the UIS measurements 
recorded from Pensacola. Ina later chapter, these findings 
will be analyzed to pinpoint any agreements or disagreements 


amongst the work groups of these organizations. 
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B. @OVERALL USER SATISFACTION 

The impressions of the respondents from Pensacola and 
Fort Ord towards their information system are comparable. 
Using a t-test, the satisfaction level of the two 
organizations, has no significant difference at alpha = 0.05. 
The Fort Ord respondents did register a slightly higher level 
of satisfaction at 10.47 which is inconsequential compared to 
Pensacola's overall satisfaction index at 9.02. Together their 
overall level of satisfaction is approximately 9.64. Figure 
8.1 describes the extent of the difference in the satisfaction 


level of the two organizations. 
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Figure 8.1 


The combined work groups' overall satisfaction was 
determined by combining the mean of the sum of their 


corresponding satisfaction indices measured from the survey 
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questions of the 57 respondents from Fort Ord and the 174 
respondents from Pensacola. The survev results, for the 
combined administrative group, displav the highest overall 

satisfaction index at 11.87, followed by the combined 
ancillary group's level of satisfaction at 9.46. Fort Ord did 
not have any survey responses from physicians. Thus the 
overall satisfaction level, -4.16, for the physicians' is 
obtained solely from the Pensacola physicians. As noted in 
chapter six, the physician's level of satisfaction has a 
statistically significant lower difference than the other 
groups at alpha = 0.05. Figure 8.2 describes the overall 
level of satisfaction of the work groups from Fort Ord and 


Pensacola. 
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Figure 8.2 
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1. Fort Ord's and Pensacola's Individual Question 
Responses 
The method used to calculate the values of the 
individual question responses was described in earlier 
chapters. Together, Fort Ord and Pensacola display a similar 
pattern of satisfaction levels for individual question 
responses as they did separately. Still, questions 1, 6 and 
11 (deals with issues concerning MID/IMD) show the highest 
level of satisfaction. Consistent with previous reports, 
questions 2, 3 and 12 (deals with DMSSC services and 
management) has the lowest rated indices. Figure 8.3 
illustrates the combined survey question responses for Fort 


Ord and Pensacola. 


Individual Question Response 
FT ORD vs PENSACOLA 
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The combined work groups displav a similar pattern of 
satisfaction levels for individual question responses as 
previously noted. Questions 1, 6 and 11 consistently rate as 
having the highest level of satisfaction, while questions 2, 
3 and 12 consistently rate as having the lowest level of 
satisfaction by the respondents of the combined work groups. 
Each question will be discussed further in the following 
sections. Since the physician's responses are solely from 
Pensacola and are not changed by combining the work group 
responses from Fort Ord, they will not be discussed as part of 
the work group. They will be reflected in the overall 
comparison of the individual question responses between Fort 
Ord and Pensacola. Figure 8.4 illustrates the satisfaction 


index for individual questions by combined work groups. 


Indıvıdual Questıon Response 
a ORD AND PENSACOLA COMBINED 


LIT LITT 
PT 
parti 
Part 
For oe 
ut. a 
DOOR OO TO 
tra er rere ser) 
a 
PS 
ETTET 


n 
NS Sa, ove 
S [i 
0... Bede 





Fa ER 
N 5 N N Y y 
EAT 
Peal | | 
bh 


Satisfaction Index 


FE 4 E x ni 
A | $ A | b 
9 
A 


f, 


— 
UN N 


NN 


102237435567 2257 371929212213 
Questıon Number 


WZ Physician RZJ Ancillary EEE Administrative 





Figure 8.4 


78 


a. Question 1 - Relationship with the Information 
Management (MID/IMD) Staff 
As noted earlier, the level of satisfaction towards 
the MID/IMD staff is relatively high. Pensacola's overall 
level of satisfaction was higher than Fort Ord's. Their 
combined administrative group and combined ancillary group 
responses to this question are practically the same. Overall, 


Pensacola shows a slightly higher level of satisfaction for 


their MID staff. There was no statistically significant 
difference at alpha = 0.05 for the combined work groups or 
hospitals. 


b. Question 2 - Processing of Requests for Changes 
to Existing Systems 
The responses to this question received the lowest 
satisfaction index rating. Both combined work groups and 
hospitals responses were negative towards the processing of 
requests for outside services. However, Fort Ord displayed 
a statistically significant lower difference lower at alpha = 
0.03: 
C. Question 3 - Degree of Training Provided to Users 
The level of satisfaction for this item was rated 
next to the lowest by both the combined work groups and 
hospitals. There was no statistically significant difference 


at alpha = 0.05. 
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d. Question 4 - User's Understanding of the System 
Fort Ord and the combined administrative group 
displayed the highest level of satisfaction toward 
comprehension of their system. There was no statistically 
significant difference between at alpha = 0.05. 
e. Question 5 - User's Feeling of Participation 
The work groups and hospitals show relatively the 
same level of satisfaction for involvement and participation 
in services and the functioning of the system. There was no 
statistically significant difference at alpha = 0.05. 
f. Question 6 - Attitude of the Information 
Management (MID/IMD) Staff 
Fort Ord the combined administrative group tend to 
show a slightly higher level of satisfaction toward their 
MID/IMD staffs. There was no statistically significant 
difference at alpha = 0.05. 
g. Question 7 - Reliability of Output Information 
Fort Ord perceives slightly higher degree of 
satisfaction for reliability of output information. The 
combined work groups' level of satisfaction seems were 
comparable. There was no statistically significant difference 


at alpha = 0.05. 
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h. Question 8 - Relevancy of Output Information (to 
intended function) 
Fort Ord and the combined administrative group had 
a higher level of satisfaction regarding the relevancy of 
output information. There was no statisticallv significant 
difference at alpha = 0.05. 
i. Question 9 - Accuracy of Output Information 
Fort Ord and the combined administrative group tend 
to be more confident in the accuracy of the output 
information. There was no statistically significant 
difference at alpha = 0.05. 
j. Question 10 - Precision of Output Information 
The precision of the output information is more 
favorable to Fort Ord and the combined administrative group. 
There was no statistically significant difference at alpha = 
0.05. 
k. Question 11 - Communication with the Information 
Management Division (MID/IMD) Staff 
The manner and methods of information exchange 
between the users and MID/IMD staff are highly valued by the 
hospitals and the combined work groups. There was no 
statistically significant difference by t-test at alpha = 


0.05. 
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1. Question 12 - Time Required for New Systems 
Development 
According to the hospitals' and combined work 
groups' level of satisfaction, the elapsed time for new 
systems development by DMSSC or contractors is practically 
unacceptable. Fort Ord may be slightly more tolerant than 
Pensacola for development of new systems. There was no 
statisticallv significant differences at alpha = 0.05. 
m. Question 13 - Completeness of Output 
Almost an even level of satisfaction amongst the 
hospitals and the combined work groups. They show some 
satisfaction towards the comprehensiveness of the output 
information. There was no statistically significant difference 
at alpha = 0.05. 
2. Combined Grouped Factors 
Fort Ord's and Pensacola's overall responses for the 
grouped factors are joined together to form overall combined 
grouped factors. These overall combined grouped factors are 
broken down to form overall work group factors. Refer to 
chapters six and seven for further discussion on group 
factors. The following sections will discuss the overall 
combined factors and combined work group factors for Fort Ord 
and Pensacola. The physicians will not be included in the 
discussion compared by work group. Neither the hospitals’ or 


work groups' satisfaction level for any of the factors were 
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statistically different at significance level alpha = 0.05. 
Figure 8.5 represents the overall grouped factors for Fort Ord 


and Pensacola. 
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Figure 8.6 represents the overall factors by work 


groups. 


GROUP FACTORS 
FT ORD AND PENSACOLA COMBINED 


x 
O 
UU 
Su 
c 
O 
= 
O 
O 
done 
n 
= 
O 
Y 


E 


Controctor 


MO 
PHYSICIAN ES23 ADMINISTRATIVE BEEY ANCILLARY 


Figure 8.6 
83 





a. Factor A (Local MID/IMD Staff and Services) 

Fort Ord and the combined ancillarv group has the 
highest level of satisfaction for the MID/IMD staff and 
services. This factor was rated the highest overall. There 
was no statistically significance difference at alpha = 0.05. 

b. Factor A (Contractor Services) 

Consistent with other results, this factor received 
the lowest level of satisfaction of anv factor. There was no 
statistically significant difference at alpha = 0.05. 

c. Factor B - Information Product 

This factor is the second highest rated of the four 
factors. There was no statistically significant difference at 
alpha = 0.05. 

d. Factor C - Knowledge and Involvement 

This factor showed no statistically significant 
difference between the level of satisfaction among the work 
groups. The administrative work group and Fort Ord rated 
knowledge and involvement slightly better than the other 


groups. 
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IX. COMPARISON OF RESULTS FROM SILAS B. HAXS HOSPITAL, 
NAVAL HOSPITAL, PENSACOLA (AQCESS) AND NAVAL HOSPITAL, 


CHARLESTON (CHCS) 


A. INTRODUCTION 
This chapter compares AQCESS and the CHCS user 
satisfaction findings. Both systems are fully described in 
chapters II and III. The survey results for user satisfaction 
are compared with respect to each site, system versus system 
and each work group. The criteria are as follows: 1) overall 
satisfaction, 2) responses to individual questions, 3) three 
group factors: MID staff and services; information system 
product; knowledge or involvement. (Baroudi and Orlikowski, 
1988) 
1. Overall Satisfaction (Pensacola, Fort Ord, Charleston) 
The mean overall satisfaction index of the combined 
sites is 11.07 on a scale ranging from -39 to +39. Charleston 
(CHCS site) displays the highest overall satisfaction at 
11.25, followed by Fort Ord at 10.47. Pensacola registers the 
least overall satisfaction at 9.025. Using a t-test, the 
level of satisfaction was not statistically significant at 
alpha = 0.05. Figure 9.1 illustrates the overall satisfaction 


index for each site. 
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2. Individual Questions 
Figure 9.2 displays the average level of satisfaction 
for each site. Responses to questions 1, 6 and 11 indicate a 


high level of user satisfaction. As stated earlier, these 
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questions are concerned with the Management Information 
(MID/IMD) staff. Consistent with previous findings, responses 
to questions 2, 3 and 12 stand out as exhibiting a relativelv 
low satisfaction index. Questions 2 and 12 are concerned with 
outside contractors and/or DMSSC services and question 3, 
training. The following discussion will further elaborate on 
each question. 
a. Question 1 - Relationship with the Management 

Information (MID/IMD) Staff 

The level of satisfaction towards the MID staff is 
relatively high (second highest satisfaction index). 
Charleston shows the highest level of satisfaction of the 
three sites, closely followed by Pensacola. Fort Ord reports 
the lowest satisfaction index. There is no statistically 
significant difference at alpha = 0.05. 

b. Question 2 - Processing of Requests for Changes 

to Existing Systems 

All three sites reports a negative level of 
satisfaction for time to process requests for changes to 
existing system. This, the lowest scoring response, is 
consistent with previous descriptions. Pensacola gave it the 
lowest rating, followed by Charleston and Fort Ord, 
respectively. The level of satisfaction for Pensacola was 
statistically significant difference lower versus Charleston's 


level of satisfaction at alpha = 0.05. 
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c. Question 3 - Degree of Training Provided to Users 
The level of satisfaction of each site was positive 
but nearly neutral at Pensacola and Fort Ord. Charleston 
reported the highest satisfaction index. However, Pensacola 
showed a statistically significant lower difference to 
Charleston's level of satisfaction at alpha = 0.05. 
d. Question 4 - User's Understanding of the System 
Fort Ord and Charleston were nearly equal in 
satisfaction, while Pensacola reported a lower satisfaction 
index. There is no statistically significant difference at 
alpha = 0.05. 
e. Question 5 - User's Feeling of Participation 
Charleston rated this item a little higher than 
Fort Ord or Pensacola, which reported a similar level of 
satisfaction. However, Pensacola showed a statistically 
significant lower difference than Charleston's level of 
satisfaction at alpha = 0.05. 
f. Question 6 - Attitude of the Management 
Information (MID/IMD) Staff 
This was the highest rated satisfaction index of 
all 13 indices. Charleston's response rated very high while 
Pensacola followed closely. Fort Ord's response was rated 
lower, but positive. Pensacola's and Fort Ord's level of 
satisfaction show a statistically significant lower difference 


to Charleston's level of satisfaction at alpha = 0.05. 
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g. Question 7 — Reliabilitu of Output Information 
Fort Ord rated this item higher than either 
Pensacola or Charleston, which responded similarly. There is 
no statistically significant difference at alpha = 0.05. 
h. Question 8 - Relevancy of Output Information (to 
intended function) 
Again, Fort Ord rated this item higher than either 
Pensacola or Charleston, which responded similarly. There is 
no statistically significant difference at alpha = 0.05. 
i. Question 9 - Accuracy of Output Information 
The response to this question resulted in a 
relatively high satisfaction index for Charleston which is 
followed by Fort Ord. Pensacola's level of satisfaction shows 
a statistically significant lower difference to Charleston's 
level of satisfaction at alpha = 0.05. 
j. Question 10 - Precision of Output Information 
Pensacola and Charleston reported similar levels of 
satisfaction. Fort Ord, although not to any large degree, 
reported a higher level of satisfaction. There is no 
statisticallv significant difference at alpha = 0.05. 
k. Question 11 - Communication with the Management 
Information (MID/IMD) Staff 
This item is rated among the top three in levels of 
satisfaction. Charleston has the highest index, Fort Ord is 


second and Pensacola, although positive, reports the lowest 
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level of satisfaction. There is no statisticallv significant 
difference at alpha = 0.05. 
1. Question 12 - Time Required for New Systems 
Development 
This item is rated as the second lowest among the 
13 responses. Charleston's response results in a negative 
index which has statistically significant lower difference at 
alpha = 0.05. Fort Ord has the highest level of satisfaction. 
m. Question 13 - Completeness of Output 
The responses to this question are relatively 
close. Charleston demonstrates the lowest level of 
satisfaction while Fort Ord has the highest. There is no 
statistically significant difference at alpha = 0.05. 
3. Grouped Factors 
The following sections will discuss the overall 
combined factors by sites. Fort Ord's and Pensacola's levels 
of satisfaction displays a statistically significant lower 
difference to Charleston's level of satisfaction at alpha = 
0.05. Figure 9.3 represents each site's level of satisfaction 
with respect to the grouped factors. 
a. Factor A (Local MID/IMD Staff and Services) 
As seen in Figure 9.3, Charleston displays the 
highest level of satisfaction, followed by Fort Ord. The 


lowest level of satisfaction is reported by Pensacola. Both 
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Fort Ord's and Pensacola's levels of satisfaction results in 
a statistically significant lower difference to Charleston's 
at alpha = 0.05. 

b. Factor A (Contractor, DMSSC) 

This factor received the lowest level of 
satisfaction of any group factor. All sites rated this factor 
extremely low, with Pensacola and Fort Ord almost rating it 
nearly neutral. There is no statistically significant 
difference at alpha = 0.05. 

C. Factor B - Information System Product 

This was the second highest rated factor. All 
three sites report similarlv, with Fort Ord recording the 
highest level of satisfaction. There is no statisticallv 


significant difference at alpha = 0.05. 
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d. Factor C — Knowledge and Involvement 
The second lowest rated factor, all sites report 
similarlv. Charleston's level of satisfaction is the highest, 
followed bv Ford Ord's. Pensacola's level of satisfaction is 
the lowest. There is no statisticallv significant difference 
at alpha = 0.05. 
4. Overall Satisfaction (CHCS vs AQCESS) 

The average overall satisfaction index of CHCS and 
AQCESS is 10.54. CHCS reports the highest overall 
satisfaction (11.25), with AQCESS at 10.13. Using a t-test, 
the difference in level of satisfaction is not statistically 
significant at alpha = 0.05. Figure 9.4 illustrates the 
overall satisfaction index for each system. 

5. Individual Questions 

Figure 9.5 displays the average level of satisfaction 
for each site. Again, responses to questions 1, 6 and 11 
indicate a high level of user satisfaction. (MID/IMD) Also 
consistent with previous findings, responses to questions 2, 
3 and 12 stand out as exhibiting a relatively low satisfaction 
index. Questions 2 and 12 are concerned with outside 
contractors and/or DMSSC services and question 3, training. 


The following discussion will further examine each question. 
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a. Question 1 - Relationship with the Management 
Information (MID/IMD) Staff 
The level of satisfaction towards the MID staff is 
relatively high (second highest satisfaction index). CHCS 
users have the higher level of satisfaction, however, there is 
no statistically significant difference at alpha = 0.05 
between the two indices. 
b. Question 2 - Processing of Requests for Changes 
to Existing Systems 
This, the lowest scoring response, received a 
negative index from both system users. CHCS users reported 
the lowest satisfaction rating, however, there was no 
statistically significant difference at alpha = 0.05. 
C. Question 3 - Degree of Training Provided to Users 
The level of satisfaction for each system was 
positive but nearly neutral for AQCESS users. CHCS showed a 
statistically significant lower difference at alpha = 0.05. 
d. Question 4 - User's Understanding of the System 
CHCS was rated higher in satisfaction than AQCESS 
for this item. There was, however, no statistically 
significant difference between the indices at alpha = 0.05. 
e. Question 5 - User's Feeling of Participation 
CHCS users reported a higher index than AQCESS 
users. AOCESS showed a statistically significant lower 


difference at alpha = 0.05. 
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f. Question 6 - Attitude of the Management 
Information (MID/IMD) Staff 
Again, this was the highest rated satisfaction 
index of all 13 indices. CHCS users' response was rated very 
high while AQCESS users, who vielded a positive rating, were 
statisticallv significant difference lower at alpha = 0.05. 
g. Question 7 - Reliability of Output Information 
The satisfaction indices for this item were 
positive and virtually the same. 
h. Question 8 - Relevancy of Output Information (to 
intended function) 
Both CHCS and AQCESS users gave positive responses. 
There was no statistically significant difference at alpha = 
0205. 
i. Question 9 - Accuracy of Output Information 
The response to this question resulted in a 
relatively high satisfaction index for CHCS users. However, 
AQCESS users show a statistically significant difference lower 
at alpha = 0.05. 
j- Question 10 - Precision of Output Information 
The CHCS and AQCESS users responded similariy with 


no statistically significant difference at alpha = 0.05. 
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k. Question 11 - Communication with the Management 
Information (MID/IMD) Staff 
Rated third highest index overall, there was no 
statistically significant difference at alpha = 0.05. 
1. Question 12 - Time Required for New Systems 
Development 
This item is rated as the second lowest among the 
13 indices. CHCS users' response resulted in a negative 
index, while AQCESS users showed a statistically significant 
higher difference at alpha = 0.05. 
m. Question 13 - Completeness of Output 
The responses to this question were positive with 
no statistically significant difference at alpha = 0.05. 
6. Grouped Factors 
The following sections will discuss the overall 
combined factors for the respondents of the CHCS and AQCESS 
systems. Figure 9.6 represents each site's level of 
satisfaction in relation to group factors. 
a. Factor A (Local MID/IMD Staff and Services) 
As seen in Figure 9.6, CHCS respondents report a 
much higher level of satisfaction for their local staff and 
services. AQCESS respondents' level of satisfaction show a 


statistically significant lower difference at alpha = Q.05. 
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b. Factor A (Contractor, DMSSC) 

This factor received the lowest level of 
satisfaction of any group factor. Both systems' users rate 
this factor low, with CHCS being statistically significant 
difference lower at alpha = 0.05. 

c. Factor B - Information System Product 

CHCS and AQCESS users responded essentiallv the 
same. There is no statisticallv significant difference in 
levels of satisfaction from either svstems' respondents at 
alpha = 0.05. 

d. Factor C - Knowledge and Involvement 

The users of the CHCS system display a slightly 

higher level of satisfaction than the AQCESS users. There is 


no statistically significant difference at alpha = 0.05. 
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X. COMBINED DESCRIPTIVE FINDINGS OF CHCS AND AQCESS 


A. INTRODUCTION 

The AQCESS and CHCS systems, although similar in many 
areas, REG functionally different systems. However, the 
survey responses from both systems are joined to assist in the 
analysis of findings across these systems. Chapter IX 
compared the Charleston Naval Hospital's CHCS system to 
Pensacola Naval Hospital's and Fort Ord Army Hospital's AQCESS 
systems. This chapter will examine the combined results of 
the survey responses from the hospitals. Figure 10.1 shows a 
breakdown of the respondents by work groups from the combined 


hospitals. 
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B. OVERALL USER SATISFACTION FOR THE COMBINED INFORMATION 

SYSTEMS 

The overall satisfaction is measured by the mean of the 
sum of the responses to the survey questions from the 
hospitals. A t-test the overall satisfaction level of the 
ancillary group and administrative group has no statistically 
significant difference at alpha = 0.05. The physicians are 
statistically significant difference lower response at alpha 
= 0.05. The survey shows that the ancillary group displays 
the highest overall satisfaction index at 11.47. When 
compared to the overall satisfaction index of the 
administrative groups level at 11.43, the ancillary group and 
administrative group overall satisfaction for their 
information systems are approximately equal. The physicians 
level of satisfaction index for the systems are the lowest at 
1.35. The overall satisfaction level was 10.14. Figure 10.2 
illustrates the combined work groups level of satisfaction for 
the AQCESS and CHCS hospital information svstems. 

i. Individual Question Responses for AQCESS and CHCS 

The method used to calculate the values of the 

individual question responses is described in earlier 
chapters. Together, the CHCS and AQCESS systems show the same 
trend as the other comparisons. Questions 1, 6 and 11 (deals 
with issues concerning MID/IMD) show the highest level of 


satisfaction. Consistent, questions 2, 3 and 12 (deals with 
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DMSSC services and management) have the lowest rated indices. 
Figure 10.3 displavs the overall level of satisfaction for 
individual questions for the CHCS and AQCESS combined work 


groups. 
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In the comparisons of one work group versus the remaining 
work groups, the patterns of satisfaction levels for 
individual question responses are similar. Questions 1, 6 and 
ll consistentiv rate the highest level of satisfaction, while 
questions 2, 3 and 12 consistently rate the lowest level of 
satisfaction by the respondents of the combined work groups. 
Each question will be discussed further in the following 
sections. Figure 10.4 displays the responses to individual by 
physicians compared to the remainder of respondents; Figure 
10.5 ancillary compared to the other groups; Figure 10.6 


administrative compared to the other groups. 


Individual Question Response 
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a. Question 1 - Relationship with the Information 
Management (MID/IMD) Staff 
As noted earlier, the level of satisfaction towards 
the MID/IMD staff is relatively high. The level of 
Satisfaction is higher for the ancillary group than the others 
but not much higher than the administrative group. The 
physicians show a statistically significant lower difference 
at alpha = 0.05. 
b. Question 2 - Processing of Requests for Changes 
to Existing Systems 
The responses to this question received the lowest 
satisfaction index rating. All work groups' responses were 
negative towards the processing of requests for outside 
services to make changes to the existing systems. However, 
the level of satisfaction of the administrative work group 
showed a statistically significant higher difference at alpha 
= 0.05. The ancillary group was much lower followed by the 
physicians. 
C. Question 3 - Degree of Training Provided to Users 
The ancillary group shows a higher level of 
satisfaction for the amount of training they have received on 
using. The physicians expressed the lowest level of 
satisfaction for the degree of training they have received. 
The level of satisfaction for the ancillary group shows a 


statistically significant higher difference and the phvsicians' 
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level of satisfaction has a statisticallv significant lower 
difference lower at alpha = 0.05. 
d. Question 4 - User's Understanding of the System 
The ancillary group displayed the highest level of 
satisfaction toward comprehension of their systems. The level 
of satisfaction for the ancillary group has a statistically 
significant higher difference at alpha = 0.05. The level of 
satisfaction for the physicians showed a statistically 
significant lower difference at alpha = 0.05. 
e. Question 5 - User's Feeling of Participation 
The ancillary group showed the highest level of 
satisfaction for involvement and participation in services and 
the functioning of the system followed closely by the 
administrative group. The ancillary group's level of 
satisfaction showed a statistically significant higher 
difference while the physicians' level of satisfaction showed 
a statistically significant lower difference at alpha = 0.05. 
£. Question 6 - Attitude of the Information 
Management (MID/IMD) Staff 
Even though the physicians showed level of 
satisfaction that was a statistically significant lower 
difference at alpha = 0.05, all work groups responded 
favorably to the attitudes of the MID/IMD staffs. The 
ancillary groups' level of satisfaction was the highest, 


followed closely by the administrative group. 
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g- Question 7 — Reliabilitu of Output Information 

The perceptions of the work groups for reliability 
of output information were relatively low. The administrative 
group's level of satisfaction was slightly higher than the 
ancillary work group's. The physicians' level of satisfaction 
was extremely low and showed a statistically significant lower 
difference at alpha = 0.05. 

h. Question 8 - Relevancy of Output Information (to 
intended function) 

Regarding the relevancy of output information, the 
administrative group had a higher level of satisfaction, 
followed closely by the ancillary group. The phvsicians' 
level of satisfaction showed a statisticallv significant lower 
difference at alpha = 0.05. 

i. Question 9 - Accuracy of Output Information 

All work groups showed positive degree of 
confidence in the accuracy of the output information. The 
level of satisfaction for the ancillary group is slightly 
higher but nearly even with the satisfaction level of the 
administrative group. The physicians level of satisfaction is 
the lowest. There was no statistically significant difference 
in the level of satisfaction for the groups. 

j. Question 10 - Precision of Output Information 
All work groups' level of satisfaction for the 


precision of the output information was relatively low. The 
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administrative group's level of satisfaction was the highest, 
followed closelv bv the ancillarv group. The phvsicians' 
level of satisfaction is the lowest. There was no 
statisticallv significant difference in level of satisfaction 
at alpha = 0.05. 
k. Question 11 - Communication with the Information 
Management Division (MIS/IMD) 
The manner and methods of information exchange 
between the users and the MID/IMD staffs was highly valued by 


the work groups. The ancillary group's level of satisfaction 


was the highest. Slightly closer were the administrative 
group and physicians. This was the highest level of 
satisfaction recorded by the physicians. There was no 


statistically significant difference at alpha = 0.05. 
1. Question 12 - Time Required for New Systems 

The work groups' level of satisfaction, for time 
required to develop new systems by DMSSC or contractors, was 
very low. The administrative group's level of satisfaction 
for the systems was the highest, followed by the ancillary 
group. The physicians' level of satisfaction was extremely 
low. The administrative group's level of satisfaction 
revealed a statistically significant higher difference and the 
physicians' level of satisfaction revealed a statistically 


Significant lower difference at alpha = 0.05. 
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m. Question 13 - Completeness of Output 
The work groups showed some satisfaction towards 
the comprehensiveness of the output information. The 
administrative group's level of satisfaction was the highest, 
followed by the ancillary group. The physicians recorded the 
lowest level of satisfaction. The administrative group's 
level of satisfaction showed a statistically significant 
higher difference at alpha = 0.05. 
2. Combined Grouped Factors 

The following sections will discuss the overall 
combined factors by work groups. The physicians' level of 
satisfaction displayed a statistically significant lower 
different for the four factors at alpha = 0.05. Figure 10.7 
represents the work groups' level of satisfaction with respect 
to the sub-total factors. Figure 10.8 displays the 
satisfaction level for the factors by the physicians compared 
to the remainder of the groups; Figure 10.9 ancillary compared 
to the others; Figure 10.10 administrative compared to the 
others. 

a. Factor A (Local MID/IMD Staff and Services) 

The ancillary group had the highest level of 

satisfaction for the MID/IMD staff, followed closely by the 
administrative group. Although this factor was rated as being 


the highest level of satisfaction by the work groups, the 
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phvsicians' level of satisfaction showed a statisticallv 


significant lower difference at alpha = 0.05. 
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b. Factor A (Contractor Services) 

The tvpe and qualitv of services rendered bv DMSSC 
or contractors have been consistently rated low by the work 
groups in all comparisons. The ancillary group showed the 
only positive level of satisfaction amongst the groups, but 
was nearly neutral. The administrative group followed with a 
negative rating for level of satisfaction and the physician 
with the lowest. The physicians' level of satisfaction showed 
a statistically significant lower difference at alpha = 0.05. 

c. Factor B - Information Product 

This factor was rated positive by all groups. The 
highest level of satisfaction was given by the ancillary group 
which was comparable to the administrative group's. The 
physicians' level of satisfaction for the information product 
was the lowest and showed a statistically significant lower 
difference at alpha = 0.05. 

d. Factor C - Knowledge and Involvement 

The administrative group's level of satisfaction 
for knowledge and involvement was the highest, closely 
followed bv the ancillarv group. The phvsicians showed a 


statisticallv significant lower difference, at alpha = 0.05. 


110 


XI. ANALYSIS OF FINDINGS 


A. INTRODUCTION 

Performance evaluation is a crucial part of the management 
of an information system. The amounts of resources invested 
and the organization's dependence on information processing 
warrant the most efficient, effective use of the information 
system. However, without direction or objective, this 
evaluation is simply an assessment of the current operation of 
the information system. (Mensching and Adams, 1991) The 
purpose of this research is to measure and document the user 
information satisfaction of the Automated Quality of Care 
Evaluation Support System (AQCESS) at the Naval Hospital, 
Pensacola, Florida and Silas B. Hays Hospital, Fort Ord, 
California. These findings from the AQCESS system will be 
compared to those from the Composite Health Care System (CHCS) 
at the Naval Hospital, Charleston, South Carolina (Hurd, 
1991). The results should not be viewed as being a conclusive 
evaluation of the AQCESS or CHCS systems, but provide a 
benchmark for analyzing user satisfaction and identifying 
possible areas of discontent. To obtain a deeper 
understanding of the identified and highlighted symptoms of 


discontent, interviews should be conducted to examine the 
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development and operating procedures of the particular work 
group involved in order to determine the underlving causes bv 
adding context and history. Upon completion of this process, 
it may become possible to make refinements and tuning to the 
system to increase its effectiveness and, perhaps secondarily, 
its efficiency. 

The first section will analyze the survey findings in 
chapters VI. through X. It will primarily focus on the 
differences and similarities among: work groups 
(administrative, ancillary, and physicians) at Pensacola; work 
groups at Fort Ord; combined work groups of Pensacola and Fort 
Ord; CHCS and AQCESS systems at Charleston, Pensacola and Fort 
Ord; combined work groups of Pensacola, Fort Ord and 


Charleston. 


B. PENSACOLA OVERALL SATISFACTION 

No one can deny that the AQCESS systems have not been a 
benefit to those sites that have this system. Automation has 
improved their overall efficiency and effectiveness. (NMDSC, 
1991). Pensacola Naval Hospital has benefitted tremendously 
from the AQCESS system. In order for the AQCESS system to 
continue to be a successful computer system, it must provide 
the needed information required by the users (Olson and 
Baroudi, 1983). The following discussion will attempt to 
analyze the levels of dissatisfaction that are statistically 


Significant by the respondents from Pensacola. Although not 
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conclusive, it should be used as a baseline measure to help 
verify areas in the AQCESS system that may require 
improvements or fine tuning. 

1. Combined Groups 

The overall Satisfaction index is a general 
description of the user satisfaction with the system. The 
Survey average satisfaction index is 9.64. The respondents 
average overall satisfaction is characterized as only being 
marginally satisfied. 

2. Comparison of Work Groups 

The physicians’ level of satisfaction showed a 
statistically Significant lower difference than the 
administrative and ancillary work groups. However, the 
administrative group was somewhat more satisfied than the 
ancillary group, but only slightly. 

The physicians’ group was dissatisfied with the 
system, which is indicated by their low average satisfaction 
index at -4.16. Generally, the only way the physicians 
legitimately interface with the system is by reviewing 
statistics of their performance, which are compiled by the 
Quality Assurance personnel. To a lesser degree, the 
physicians interact with the system through the Ad Hoc Report 
Generator module in the system. The physicians mainly view 
the system as a means of reporting their performance of 


patient care, which, according to their perspective, may not 
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be accurate. Therefore, anv inaccuracies in a phvsician's 
performance report is viewed negativelv bv that phvsician and 
potentiallv leads to conflict with those who generate the 
reports. So, it is reasonable to assume that their overall 
low level of satisfaction can be attributed to their limited 
interface with the AQCESS svstem. 
Ancillarv made up 39 percent of the personnel surveved. 
Their average satisfaction index is 11.49, slightly lower 
than the administrative group. Most of the users in this work 
group are technicians who use the system for direct medical 
care (e.g., canceling and editing disposition data, patient 
history, diagnosis and procedure data record tracking and 
reporting). As the large number of items processed each day 
in these areas increase, the response time for processing 
decreases to unacceptable levels. It has become routine for 
ancillary personnel to wait more than five minutes to retrieve 
records. When time is critical for patient care, this waiting 
is a source of frustration for ancillary personnel. 

The administrative group recorded the highest overall 
satisfaction score. Administrative personnel use the system 
for purposes other than direct conduct of patient care. With 
a few exceptions, the work performed on the system can be 
accomplished without the pressure of a limited and specific 
time constraints. Therefore, they can use the computer system 
at a comparatively slower pace than the other work groups. 


Consequently, interviews with most administrative personnel 
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revealed concern about the slow computer response time. 
Additionallv, interviews revealed a lack of trust for the 
integritv of the output information bv the svstem. Some 
administrative personnel argued that their inability to 
correct data, once it had been entered into the svstem, made 
the information suspect. 
3. Significant Individual Question Differences and 

This section will discuss only those responses to the 
questions with significant differences among the work groups. 
The physicians' level of satisfaction for all questions showed 
statistically significant lower differences than those of the 
other work groups. A low level of satisfaction for all 
questions by the physicians can be expected due to their lack 
of involvement and understanding of the system coupled with 
their limited role as a user of the system. Those questions 
that are statistically significant for the ancillary and 
administrative personnel will be discussed in the following 
sections. Since the physicians' responses to the questions 
are statisticallv significant in every case, only those 
questions, which will be helpful in gaining a better 


appreciation of the degree satisfaction, will be analyzed. 
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a. Relationship with the Management Information 
Department (MID) 


The results of the survev showed the phvsicians to 
be the least satisfied with the services of bv the MID staff. 
Ironicallv, their level of satisfaction for this question was 
the highest of all their responses. Typically, unless it is 
to report inaccuracies in their performance report, which is 
generated by the system, the physicians' seldom communicate 
with the MID staff. If this was the only method in which the 
physicians and the MID staff communicated, then the 
physicians' relatively low level of satisfaction with the 


staff can be expected. 


b. Processing of Requests for Changes to the Existing 

System 
DMSSC is responsible for providing the support and 
services for the AQCESS system. Request for changes must go 
to DMSSC for approval and funding. If enough AQCESS sites 
request or support the change, then an upgrade to the system 
will be made. This upgrade, after testing, is then released 
as an improved version of the previous AQCESS system. The 
latest version for testing is AQCESS 7.0, which will take a 
year to release, if successful. (ADPAQCESS NMDSC, 1991) 
Because AQCESS is under centralized control (DMSSC), changes 


to AQCESS will not occur rapidly. 


116 


Cc. User's Understanding of the System 
Users with better computer background and training 
are likely to be more confident in their ability to use the 
system and are more satisfied with the experience. The work 
groups that reported a low level of satisfaction in user 
training (physicians in particular) demonstrated a 
significantly lower level of satisfaction with their 
understanding of the system. 
d. User's Feeling of Involvement 
The presumption is that user involvement will lead 
to better understanding of the system and develop a system 
tailored to meet specific needs. Therefore, users will be 


more satisfied with the system than if they had not been 


involved. (Baroudi, Olson and Ives, 1986) Physicians are 
rarely involved with the system. So, their low of level 
satisfaction is expected. Consequently, administrative 


personnel are involved daily in the operations of the system 
and they feel involved with the system. 
e. Relevancy of Output Information 
when the output information is less than what the 
user perceives, the user will be less satisfied with the 
system (Conrath and Mignen, 1990). All groups have a low 


perception of the output information. 
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f. Accuracy of Output Information 

Physicians often question the accuracy of their 
statistical performance reports. If the information is 
inaccurate, then the system may be more likely to be blamed 
for the mistake than the person who inputs the information. 
This may be the main source of discontent for the physicians. 

g. Completeness of Output 

The comprehensiveness of the output information 
relates to the relevancy and accuracy of the information. If 
the user perceives the information to be lacking, the user 
will more likely think the information is inaccurate and/or 
irrelevant. It is not surprising to find the administrative 
group scoring significantly higher than the other groups on 
this question. The administrative group does not use this 
information directly. 
4. Grouped factors 

The overall management of the AQCESS system is 
centrally controlled by DMSSC who are responsible for the 
software support for the system. Local support services are 
provided by the local MID. Therefore, the single factor, MID 
staff and services, found in the original study by Ives, 
Olson, and Baroudi (1988), was altered to make two separate 
factors: local MID staff and services and Contractor's 


services. 
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The phvsicians scored the four factors statisticallv 
significant lower and the administrative group scored the four 
factors statisticallv significant higher for level of 
satisfaction. As alluded to earlier, due to the physicians' 
limited role and involvement with the system, their low level 
of satisfaction is predictable. The administrative group is 
not as constrained for time as the ancillary group. They are 
more involved with the system and may have a higher level of 
satisfaction, but not significant. 

It is noteworthy to examine the significant positive 
level of satisfaction towards the local MID staff and 
services. Although there was a statistically significant 
lower difference, the physicians' level of satisfaction for 
this factor was positive, the only factor to be scored 
positive by the physicians. The administrative and ancillary 
groups' satisfaction index indicated that they were very 
satisfied with the services provided by the local MID staff. 
Common sense would suggest that the level of satisfaction is 
directly related to effective communication (Conrath and 
Mignen, 1990). Thus, it can be reasonable to assume that the 
MID staff is effectively communicating with the ancillary and 
administrative group and, to a lesser degree, the physicians. 

Bailey and Pearson (1983) listed contractor services 
and time required for system changes as the two most frequent 
elements for causing dissatisfaction among users. The outside 


services rendered by DMSSC were viewed unfavorably by the work 
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groups. DMSSC oversees 166 sites DoD wide. Each site has an 
AOCESS Customer Support Representative who is responsible for 
providing direct support of that user. A site has several 
means of reporting their problems; E-mail, System Change 
Requests (SCRs), work shops etc. (NMDSC, 1991) Any requests 
for changes to a system or its software, unless the change is 
critical, will be a lengthy process. Since most of the users 
are not familiar with the centralized process or DMSSC, it is 
only reasonable to expect the low level of satisfaction for 
this factor. 

It appears that the users will be satisfied if the 
output information is accurate, comprehensive, reliable and 
relevant, the users' will be satisfied. The work group was 
slightly satisfied with the factor regarding the quality of 
output information delivered by the AQCESS system. 

Powers and Dickson (1973) argues that no matter how 
effective a system might be, if it is not perceived to be 
satisfactory, it will be underutilized. AOCESS is training 
intensive and is not designed for the computer novice. 
Consequently, some of the features of the system are 
underutilized. (NMDSC, 1991). If that is the case, then a low 
level of satisfaction will be displaved bv the users. The 
administrative and ancillarv groups displaved a nearlv neutral 
satisfaction index for the knowledge and involvement factor. 
The satisfaction index was negative for the phvsicians. The 


administrative group exhibited a satisfaction level which 
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showed a statisticallv significant higher difference than the 
other two groups. The administrative group can be considered 
slightly satisfied with this factor. 

5. Satisfaction versus Time of System Use 

In a study conducted by Igbaria and Nachman (1990), a 
significant relationship was found between user satisfaction 
and system utilization. This study showed an increase in 
overall satisfaction as time of system use increased. Then, 
the level of satisfaction decreases after extended use 
(greater than 11 months) of the system. 

Initially, new personnel are preoccupied with learning 
the AQCESS system in order to learn their job. Once they have 
become comfortable and familiar with the AQCESS system, the 
users are more concerned with doing their job. Consequently, 
the users become more intolerant to system down time or slow 
response time. Also, as their knowledge of and experience 
with the system grows, they can more easily comprehend the 
reasons for system problems and only wonder why the experts 
can not fix them. Thus, their level of satisfaction for the 


system has decreased with time. 


C, FORT ORD OVERALL SATISFACTION 
A comparison of the satisfaction levels of the respondents 
from the Silas B. Hays Hospital will provide a baseline 


measure for future evaluations of the system. 
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l. Combined Groups 

Fort Ord's average satisfaction index is 10.47. By 
referring to the index ranges of -39 to 39, the respondents' 
average overall satisfaction is classified as being only 
slightly satisfied. 

2. Comparison of Work Groups 

There were two work groups identified in this study, 
administrative and ancillary, instead of three because no Fort 
Ord physicians responded to the survey. (A spokesman stated 
that the physicians did not interact with the system.) There 
was no statistically significant difference between the work 
groups. However, the ancillary group was more satisfied than 
the administrative group. One problem with this assessment is 
that the ancillary group's surveyed population only composed 
approximately three and one half percent of the total 
respondent population as opposed to the administrative group's 
overwhelming 96.5 percent. Therefore, the ancillary group's 
results may not be an accurate representation of the actual 
group population as a whole. 

Both administrative and ancillary personnel had very 
low regards for the timeliness of outside services and 
modifications to the system, as well as the amount of training 
they received. However, their collective satisfaction with 
the local IMD staff was considerably higher. The users 


appeared to recognize that the local staff was both willing 
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and capable of performing services which were not governed bv 
higher authoritv. 
3. Significant Individual Question Differences and 

Grouped factors 

There was no statistically significant difference 
between the work groups for any of the responses to the 
individual questions or grouped factors. The ancillary group 
generally appeared to be more satisfied in the individual 
question areas as well as the four grouped factors. However, 
as suggested earlier, this may be attributed to the ancillary 
group's size. 

4. Satisfaction Versus Time of System Use 

The ancillary group only had users with six to ll 
months of system use, however, their level of satisfaction was 
relatively high. The administrative group's level of 
satisfaction actuallv increased as the respondents' time of 
svstem use increased. This trend is consistent with that 
which was predicted in chapter VII. Manv respondents equated 
experience with the system to being able to efficiently and 
effectively manipulate the system and further translated that 


to "job satisfaction". 


D. PENSACOLA AND FORT ORD OVERALL SATISFACTION 
A comparison of the satisfaction levels of the respondents 


from the both systems will further aid in future evaluations. 
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1. Combined Groups 
Combining both Pensacola and Fort Ord 's work groups 
produces an average satisfaction index of 9.64. The results 
indicated that there was a low level of satisfaction among all 
AQCESS respondents. 
2. Comparison of Work Groups 
The work groups identified in this study, included 
both Pensacola's and Fort Ord's administrative and ancillary 
groups. The administrative group displayed the highest level 
of satisfaction, followed by the ancillary group. It has been 
established that the administrative group's use of the 
information is less critical than that of the ancillary group. 
As a result, they are less frustrated with slow response and 
down time of the system. 
3. Significant Individual Question Differences and 
Grouped factors 
The combined work groups at Pensacola showed a 
statistically significant higher difference than Fort Ord for 
question 2. It would appear that neither site is at all 
satisfied with the way in which outside contractors or DMSSC 
process requests for changes to the system. Some users 
expressed dissatisfaction over not having their requests 
honored in a timely manner. Others understood and agreed with 
the process involved in implementing changes. After all, if 


all requests were acknowledged and implemented, there would 
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probablv be a number of unique AQCESS svstems throughout Dop. 
There was no statisticallv significant differences regarding 


the grouped factors. 


En FORT ORD, PENSACOLA AND CHARLESTON OVERALL SATISFACTION 

Current plans for AQCESS include many new initiatives. 
AQCESS 6.0 was tested and accepted by representatives from the 
three services in March, 1991. Expanded functions have been 
incorporated into AQCESS 7.0 which include scrolling help 
windows with "point and shoot", data entry from help tables 
and Ad Hoc Reporting. Beta testing is expected to begin the 
during the first quarter of fiscal year 1992. DoD funding has 
been established to support hardware upgrades at selected 
sites in fiscal years 1992 and 1993. Despite all these 
upgrades to the AQCESS system, scheduling in September and 
December of 1992 the AQCESS systems at Pensacola and Fort Ord 
will be replaced by the CHCS system. (NMDSC, 1991) 

A comparison of the satisfaction levels of the respondents 
from the combined AQCESS systems and the CHCS system will aid 
in future evaluations. 

1. Combined Groups 

The survey average satisfaction index is 9.64. 
Charleston is more satisfied with their system than Fort Ord 
or Pensacola. Pensacola shows a less than average level of 
satisfaction. This may be attributed to the fact that the 


Pensacola physicians displayed a very low level of 
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satisfaction for the AQCESS svstem. The differences in the 
levels of satisfaction were not statisticallv significant. 
2. Significant Individual Question Differences and 
Grouped factors 
This section will only discuss those questions that 
were statistically significant. 
a. Processing of Requests for Changes to the Existing 
System 
All sites responded negatively. Charleston is less 
satisfied than the other sites and happens to be a beta test 
site for the CHCS system. Software support for this site is 
furnished bv a civilian contractor. The CHCS system is a 
relatively new system under development, whereas the AQCESS 
system has been operational since 1984. The developers and 
service personnel of the AQCESS system have had the 
opportunity to identify and correct many software problems. 
The AQCESS systems' MID/IMD personnel are less concerned with 
system implementation and more concerned with system 
maintenance, so there are fewer requests for changes. 
Therefore, the users of the AQCESS system may be less 
frustrated. 
b. Degree of Training Provided to Users 
Charleston was more satisfied with their level of 
training than the other sites. Since Charleston was a test 


site and was in the implementation phase, the users' formal 
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training was more extensive than the other sites. Therefore, 
Charleston was expected to show a higher level of 
satisfaction. 
c. User's Feeling of Participation 
Charleston was more satisfied with their 
involvement with the CHCS svstem than the users of the AQCESS 
system. This can be expected, especially when comparing the 
physicians of both sites. The CHCS physicians use the system 
for direct patient care. In fact, the CHCS physicains have a 
terminal located in their office. Whereas, the AQCESS 
physicians' only legitimate way of interfacing with the system 
is by reviewing statistics of their performance compiled by 
the Quality Assurance personnel. 
d. Attitude of the Information Management Staff 
Charleston was more satisfied with the Information 
Management staff. Since they are a test site, the MID staff 
provided more recent and extensive training. The 
communication between the CHCS users and their staff was 
expected to be more satisfying. 
e. Accuracy of Output Information 
The accuracy of output information for the CHCS 
physicians, in many instances, is directly related to the 
information that they input. The AQCESS physicians have 


little or no control over the input information. Therefore, 
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it is reasonable to assume that the level of satisaction for 
CHCS users will be higher. 
f. Factor A (Local MID/IMD Staff and Services) 

Charleston shows a higher level of satisfaction. 
The CHCS users' perception of the local MID is considered to 
be quite satisfactorv. There does not appear to be any 
communication problems between the CHCS users and the MID 
staff. The AQCESS users appear to be satisfied with the level 
of service from their MID/IMD staff, but to a lesser degree. 
The higher level of satisfaction for the CHCS group can be 
attributed to the increased communication required between 
them and their MID staff while undergoing testing of the 


system. 


F. COMBINED CHCS AND AQCESS OVERALL SATISFACTION 
The work groups from the AQCESS and the CHCS svstems have 
been combined to pinpoint problems of a particular work group. 
The following sections will analvze the responses from the 
combined work groups. 
1. Overall satisfaction 
The overall satisfaction index was 10.14. This places 
the overall satisfaction index in the lower third of the 
scale. This score should be used as a baseline measure for 
follow-on analysis of these hospital information systems. 
The ancillarv and administrative groups, with 


comparable satisfaction indices, appear to be slightly 
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satisfied with their information svstems. The phvsicians, 
with a near neutral satisfaction index, can be considered to 
be unsatisfied with the svstem. 
2. Significant Individual Questions 
The following section will only address those 
questions with statistically significant differences among the 
work groups. The individual questions will be combined into 
grouped factors and discussed in the next section. 
a. Processing of Requests for Changes to Existing 
Systems 
The administrative group shows a higher level of 
satisfaction. However the results show a neutral satisfaction 
index for the administrative group and negative satisfaction 
indices for the ancillary group and physicians. This implies 
that the work groups are dissatisfied with the contracted 
services, which is consistent with previous analysis. 
b. Degree of Training Provided to Users 
The physicians consistently demonstrated a lower 
level of satisfaction for training in all the previous 
comparisons. Clearly, this is an area for further study. 
c. User's Understanding of Systems 
The low level of satisfaction is evident among all 
work groups and, in particular, the physicians. This is 


consistent with previous findings that suggests users with 
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better computer background and training are more likelv to be 
more satisfied with the svstem. 
d. User's Feeling of Participation 
As previously noted, the users who are more 
involved with the system gain a better understanding of the 
system. As a result, are more satisfied. In the case of the 
AQCESS physicians, they are rarely involved with the system. 
Contrarily, the CHCS phvsicians with terminals in their 
offices, may find the system to be a distraction from their 
daily routine. Thus, the physicians’ negative satisfaction 
index indicates their dissatisfaction for their current level 
of participation. 
e. Attitude of the Information Management Staff 
For this item, the physicians indicated 
satisfaction with the system, even though they displayed a 
lower level than the other groups. This is the highest 
satisfaction index recorded by the physicians. The 
administrative and ancillary groups were highly satisfied with 
the attitude of their MID/IMD staffs. 
f. Reliability of Output Information 
The physicians are not satisfied with the 
reliability of output information. This can be attributed to 
their roles and involvement with their systems. The 


administrative and ancillary personnel are only slightly 
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satisfied. If the information is inconsistent, the users will 
have a low level of satisfaction. 
g. Relevancy of Output Information (to Intended 
Function 
The physicians perceived the output information as 
not being what they required. The administrative and 
ancillary groups were reasonably satisfied. 
h. Time Required for New Systems Development 
Contractor services and time required for system 
changes are viewed unfavorably by the work groups. Responses 
to requests for changes to information systems that are 
controlled by a central management (DMSSC) are more likely to 
be slower than responses for an information system that has a 
decentralized management. 
i. Completeness of the Output Information 
All groups show a positive satisfaction index. 
However, consistent with previous analysis, the physicians 
perceived the information to be incomplete. If the 
information is perceived to be inaccurate, irrelevant and 
inconsistent, users will not be satisfied with the output 
information. 
3. Comparison of Grouped Factors 
The physicians are significantly less satisfied for 
all factors. That could be, for the most part, due to 


combining the CHCS physician responses with those of the 
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AQCESS physicians. Although the sample populations for the 
two physician groups differ by only one, the extremely 
negative scores reported by the Pensacola group significantly 
influenced the overall low scores of the combined physician 
group. "Nevertheless, the physicians tend to show a positive 
level of satisfaction for the MID/IMD staff and services along 
with the administrative and ancillary groups. 

The analysis of the responses form the work groups for 
contractor services and time required for system changes are 
consistent with other analysis. The administrative and 
physician groups view the contractor services negatively, 
while the ancillary group is neutral. This was the most 
frequent cause of dissatisfaction for the users. 

The administrative and ancillary groups were slightly 
satisfied with the quality of output delivered by the system. 
The physicians, however, were dissatisfied. 

The physicians were dissatisfied with knowledge of and 
involvement with the system. If users are lacking in 
training, understanding, and experience with the svstem, thev 


are more likelv to be less satisfied. 
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XII. CONCLUSIONS AND RECOMMENDATIONS 


A. MEASURING USER INFORMATION SATISFACTION 

Today, there is a greater need to understand the user's 
point of view. As the availability of hardware and software 
continues to grow, it becomes more difficult to remain current 
with the pace of technological changes. Because of personal 
computing, users are far more knowledgeable. They are less 
tolerant of complex time-consuming processes and procedures, 
delays due to backlogs and shifting priorities and inadequate 
procedures and information on available resources. (Conrad and 
Mignen, 1990, Cash et al, 1988, Mensching and Adams, 1991) 
Conrad and Mignen (1988) argue that the focus should not be on 
the cure for user dissatisfaction, but to develop the means to 
identify problems before they occur. This is almost 
impossible without some means of gauging users' perceptions. 
The user information satisfaction survey questionnaire 
provides the means for gauging users' perceptions. The survey 
questionnaire is easily and quickly administered and provides 
a standard measure for comparisons of scores across 
departments, systems, users, organizations, and industries. 
This survey is not a conclusive evaluation of the AQCESS and 


CHCS systems, but coupled with further investigation, can be 
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a powerful tool in the analvsis and interpretation of the 


causes of user dissatisfaction. 


B. SATISFACTION CHARACTERISTICS OF AQCESS 

Overall satisfaction is a generalization of all the 
characteristics that effect the satisfaction of the user. 
Along with overall satisfaction, the study specifically looked 
at the four factors that make up satisfaction. 

1. Overall Satisfaction 

The survey results indicate the users are slightly 

satisfied with AQCESS at Silas B. Hays Army Hospital, Fort 
Ord, and by Naval Hospital, Pensacola. The phvsicians' 
responses were consistently scored low across CHCS and AQCESS. 
The lack of Fort Ord physician responses to the survey may 
have caused a slightly inflated overall satisfaction score for 
them. The overall satisfaction scores across organizations 
and systems did not differ significantly. 

2. Areas of Satisfaction 

All three work groups (to a lessor extent the 

physicians) rated the local MID/IMD staff and services as 
being satisfactory. However, the CHCS users' satisfaction 
index is twice that of the AQCESS users'. This suggests that 
the local AQCESS MID/IMD personnel are not communicating and 
interacting with their users as effectively as the CHCS MID 
personnel are with their users. CHCS's success in this area 


can be attributed to the emphasis on training the users during 
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the implementation phase of the CHCS system. The AQCESS 
systems' MID/IMD personnel, far beyond the implementation 
phase, have no need to intensify the training of their users. 
However, they do maintain a solid steady training program that 
includes both formal and on-the-job training. 

In many areas, the AQCESS svstems' administrative 
groups displaved higher satisfaction than anv other group. 
The ancillarv groups, whose interaction with the svstem is 
more essential (regarding direct patient care), were more 
dissatisfied because of the svstem's response time. 

3. Areas of Dissatisfaction 

Armed with a set of problem areas, the MID/IMD staff 
and functional managers of the AQCESS svstems can explore the 
underlining causes through: (1) interviews and (2) bv 
examining the development and operating procedures of 
particular user groups within their systems. Investigation 
into areas of dissatisfaction can provide context, history and 
insight for possible corrective measures. 

Since the physicians' satisfaction indices were low, 
across the organizations, it seems to be a characteristic of 
physicians to display a low level of satisfaction for hospital 
information systems. The limited involvement of the Pensacola 
physicians was a major factor for their overall low level of 


satisfaction. 
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The respondents were most dissatified with the amount 
of time it took for new svstem development and changes to 
OEELT: AQCESS is a centrallv managed information svstem, 
therefore it requires a relativelv long time to process 
requests for changes. 

4. Satisfaction with Time of Svstem Use 

Regression analvsis revealed no correlation between 
satisfaction and time of system use. The Fort Ord 
administrative group displayed increased satisfaction with 
increased time of use. All Pensacola groups, with greater 


than 11 months of experience, were less satisfied. 


C. RECOMMENDATIONS 

The short-form questionnaire developed by Baroudi and 
Orlikowski (1988) is an effective means to measure user 
information satisfaction. This survey instrument is an 
appropriate instrument to document user satisfaction within 
Medical Treatment Facilities as well as documenting changes in 
user satisfaction at Naval Hospitals, Pensacola and Silas B. 
Hays (Army) Hospital at a later time. 

This type of survey, to prevent biased results, is best 
conducted by a researcher who is not experienced with the 
system. Thus, the arten of the results are more 
likely to reflect impartiality. It is recommended that the 
researcher arranges with a member of the organization to serve 


as a point of contact. The contact distributes and collects 
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the questionnaires for the researcher. It should be 
emphasized prior to the survev that the questionnaire elicits 
responses reflective of the present conditions and not the 
past conditions and experiences with information svstems or 
MID/IMD staffs. In employing the user satisfaction 
questionnaire, it was found that some individuals were 
uncertain as to the exact meaning of certain questions. If 
the lack of clarity is likely to be a problem, it is 
recommended that full explanations of the scales be included. 
Additionally, direct contact with individuals for interview by 
the researcher is beneficial for adding context to history. 
Additionally, the following is recommended prior to conducting 
the interview: 

1. Structure the interviews around known problem areas to 
avoid "orienting" time spent in searching for real 
issues. 

2. Avoid focusing on the highly specific, idiosvncratic 
complaints of individual users that are not of general 


concern. 


3. Reduce the number of interviews needed to obtain deeper 
undertsanding of the problem areas. 


To obtain a deeper understanding of the issues bv adding 
context to historv, it is also recommended that the following 
items be further investigated: 


1. The administrative group's overall higher satisfaction 
over the other work groups. 


2. The phvsicians' lowered perception of satisfaction in 
almost all areas. 


3. The reasons for the respondents having negative 
perceptions of satisfaction with contractor's services. 
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The user satisfaction questionnaire should be used as a 
standard measuring tool. This tool will allow both the 
researcher and the practitioner to utilize a readilv available 
instrument, thus, avoiding the process of developing a new 
measure each time an assessment of user satisfaction is 
required. A follow on study should be conducted at Silas B, 
Hays Hospital, Fort Ord and Naval Hospital, Pensacola, using 
the results of this study as a baseline comparison. Finally, 
the results of this study should be used for measuring user 


satisfaction at other DoD hospital sites. 
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APPENDIX A 


Part A: General Information 


Hospital Division/Department: (Check one) 





General Administration 
Nursing Administration 
Dietarv 

Emergency 

Laboratory 

Medical Clinic 
Inpatient Nursing 
Pharmacy 

Radiologv 

Other (Specifv): 


Job Description: (Check one) 


Administration 
Medic 

Technician 

Nurse 

Pharmacist 
Phvsician 
Phvsician Assistant 
Other (Specifv): 


Highest Level of Education: (Check one) 





Age: 


High School Graduate 
Some College 
Associate's Degree 
Bachelor's Degree 
Some Graduate Work 
Master's Degree 
Doctoral Degree 
Medical Degree 
Other (Specifv): 


Gender: Male Female 


Length of time (in months) vou have used AQCESS: 


Have 


If your answer was Yes to question 7, was it a health care 


you used other computer systems before ? 


information system ? Yes No 
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Part B: User Satisfaction Questionnaire 


This section convevs vour personal feelings concerning the use 


of the Automated Qualitv of Care Support Svstem at Naval Hospital, 
Pensacola / Silas B. Hays Hospital (Fort Ord). Please do not 
attempt to analyze the questions. Remember, there are no right or 
wrong answers. 


Please follow these instructions: 


ilu 


4. 


insufficient : e 


Check each scale in the position that describes your 
evaluation or answer. Example: Inventor of the traffic light 


genius : 1 : : > : : 3 : incompetent 
You would check an "X" in the left-hand side of the scale 
(marked as "1" in the above example) if you thought the 
inventor was a genius. Similarly, you would mark the 
mid-point of the scale ("2" in the above example) if you 
thought the inventor was neither a total genius nor totally 
incompetent and the right-hand side (point "3" in the above 
scale) if you thought the inventor was incompetent. The other 
positions can be used to indicate varying degrees between 
$" 1 n and ti 2 UL or et 2 e and ef 3 ti - 


Check in the space, not between spaces (be sure to 

check only one position per scale). 

(Correct way > : X : Incorrect way => : x = 

Check both of the scales after each question. 

work rapidly, do not omit any questions and rely on your 


first impressions. 





ANSWERS BASED ON YOUR OWN OPINIONS 


Relationship with the Management Information Department (MID) 
Information Management Division (IMD) staff 
dissonant : : : : : : 


harmonious 
good 


bad : : 


Processing of requests for changes to existing systems 
fast : : : : : $ : slow 
untimelv : : timely 





Degree of training provided to users 
complete : : : : 
low : 


incomplete 
high 


User's understanding of systems 
sufficient 
incomplete 


complete : : 


5. User's feel 
positive : 
insufficient : 
6% Attitude of 


Management 
cooperative : 
negative : 

7. Reliabilitv 
high 
superior 


ee oo 


8. Relevancv 
useful : 


relevant 


ee 


9. Accuracv of 
inaccurate : 


low : 
10. 


low 
definite 


EL. 


ing of participation 


Dr en * =—=srn£ÈÒm_—-_—_—_t _———eel $ 


tiIT_—_—_—— e" Ne —=< "_ ‚e A °"0"0@" 0} 


the Management Information 
Division staff 


Sn EE ee em 


of output information 


me - È‘ 


output information 


——— ë: ë a ë M: ëM es 


Information Management Division staff 


dissonant : 
destructive : 


12. Time requi 
unreasonable : 
acceptable : 
13. Completene 
sufficient : 
adequate : 


red 


— LEE 


ss of the output information 


E OO el 


negative 


sufficient 


Department / Information 


belligerent 
positive 


low 
inferior 


function) 


Communication with the Management Information 


useless 
irrelevant 


accurate 
high 


high 
uncertain 
Department / 
harmonious 


productive 


reasonable 
unacceptable 


insufficient 
inadequate 


Thank you again for your cooperation 
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APPENDIX B 


Automated Quality of Care Evaluation 
Support System Surve 


This survey is part of a study of the Automated Quality of Car 
Evaluation Support System (AQCESS) and its effectiveness in 
military hospital settings. The purpose of the study is to obtain 
information regarding your perceptions or views of how well AQCESS 
functions in your specific area of use. 


Just a few minutes are required to fill out general informatio 
along with a 13 question survey. Your identity will remain 
confidential. The data gathered through the survey will provide 
valuable insight into the system's strengths and/or weaknesses as 
well as assist in future research, development and training. 


The success of this survey depends on my receiving as many 
completed surveys as possible from users in all hospital areas. 
Your participation lends an important contribution towards this 
end. There are no right or wrong answers, only your candid 
response to each question. Please do not omit any of the question 
in either section of the survey. 


Please return your Completed survey to LCDR Neeley in the 
Management Information Department / Mr. G. Scott (HSXT-IMD) in the 
envelope provided. Thank you for your cooperation. 


James R. Booth, LCDR, USN / John L. Bryant, Jr., LT, US 
Naval Postgraduate School 

SMC 1073 / SMC 2604 
Monterey, CA 93940-4831 
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APPENDIX C-l 


NAVAL HOSPITAL PENSACOLA 


COMBINED RESULTS FOR PHYSICIANS VS ANCILLARY VS 


ADMINISTRATIVE 

TOTAL SATISFACTION AVERAGE 
PHYSICIAN -0.3899 
ADMIN 1.42659 
ANCILLARY 0.72624 
GROUP FACTORS A 

(MID) 

PHYSICIAN SAT 0.64394 
ANCILLARY SAT 1.41176 
ADMIN SAT 12 7123 
PHYSICIAN STDEV 1.430613 
ANCILLARY STDEV 1.52979 
ADMIN STDEV 1.44426 


INDIVIDUAL QUESTION RESPONSES 


QUESTION 1 

PHYSICIAN 0.63636 
ANCILLARY 175 
ADMIN 1.72024 
STAN DEV 1 

PHYSICIAN 1.58245 
ANCILLARY 1253872 
ADMIN 1.40121 
QUESTION 6 

PHYSICIAN 0.5 
ANCILLARY 1.39706 
ADMIN 1.88095 
STAN DEV 6 

PHYSICIAN 1.58831 
ANCILLARY 1297372 
ADMIN 1.41762 
QUESTION 11 
PHYSICIAN 0.79545 
ANCILLARY 1.33824 
ADMIN 14939471 
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INDEX 

-4.15909 
12.83929 
11.49265 


A 
(CON) 


-0.76136 
0.003676 
0.309524 
1.538883 
1.694494 
1. 76929 


2 
-1.04545 
-0.09504 
0.309524 


2 
1.445283 
1.800855 
1.863362 


7 
-0.61364 
0.941176 
0.952381 


7 
17599283 
1.670526 

1.76544 


12 
-0.47727 
0.154412 
02213178 


72074955 
. 70882 
-00357 
. 93888 
73813 
169909 


Herero 


3 

-1 
0: 13971 
0.19048 


3 
1.43019 
1.335968 
1281578 


8 
-0.6591 
0.91176 
1730337 


8 
1.46075 
1207303 
152208 


13 
-0.4091 
0.72794 
1.10714 


-1.01136 
0.376838 

0259375 
1.595408 
1.816868 
1.828224 


4 
-0.75 
0.69117 
079985. 


4 
1.79804 
172530 
1.80394 


9 
-0.4772 
0.61029 
0797023 


9 
1.42204 
A OZ 
1272318 


5 
-1.25 
0282333 
0.91667 


b 
1.03936 
1.67104 
1202053 


10 
023192 
0.35294 
0.68452 


10 
1.60642 
1.082065 
1.201233 


STAN DEV 11 12 13 


PHYSICIAN 1-0957 1.097283 1.6412 
ANCILLARY 174714 1.566556 1.85293 
ADMIN 1.49161 1.647337 1.85293 
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APPENDIX C-2 


NAVAL HOSPITAL PENSACOLA 


PHYSICIANS VS THE REST OF THE SURVEY GROUP 


GENERAL AVERAGE INDEX 
PHYSICIAN -0,3899 -4.1591 
STDEV 1.62264 9.4067 
NON-PHYSICIAN 0.8707 13.7796 
STDEV 17921677207355 

GROUP FACTORS A A 

(MID) (CON) 
PHYSICIAN GROUP 0.64394 -0.76140 
NON-PHYSICIAN 137/7897 0:17270 
PHYSICIAN STDEV 1.436018 1.31421 
NON-PHYSICIAN STDEV 1.49064 1.74287 

QUESTION 1 2 
PHYSICIAN SAT 0.63636 -1.04550 
PHYSICIAN STDEV 1.58245 1.44528 
NON-PHYSICIAN SAT 1#62171720270526 
NON-PHYSICIAN STD 1.46841 1.86081 

QUESTION 6 Z 
PHYSICIAN SAT 0.50000 -1.0455 
PHYSICIAN STDEV 1.58831- 1.55528 
NON-PHYSICIAN SAT 1.66447 0.94737 
NON-PHYSICIAN STD 14 530850 1.72363 

QUESTION 11 12 
PHYSICIAN SAT 0.79545 -0.47730 
PHYSICIAN STDEV 1.05700771.09728 
NON-PHYSICIAN SAT 1.44737 0.24013 
NON-PHYSICIAN STD 1.48585 1.61354 
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-0.49550 


-l. 
1: 
O. 
l, 


Ra a a 


oro 


B 


087171 
0.28359 
L22901 


3 
0000 
4301 
1677 
8357 


8 
¿0000 
.4607 
11282 
26038 


13 

. 4091 
.61412 
.8914 
378611 


ee oO 


hOro 


E 
LA 
0.49671 
0331790 
1.82634 

4 5 
.7500 = 132500 
738077263936 
.8388 0.87500 
. 7872 1.67140 

9 10 
.6136 -0.3182 
.4220 1.49033 
. 8092 0.53618 
-7386 170101 


APPENDIX C-3 


PHYSICIANS vs THE REST OF THE SURVEY GROUP 


CALCULATED T-VALUE 


Ji 2.91074* 

2 2.7808T2 

3 2.86054* 

4 3.89161* 

9 5.734355 

6 3.34561* 

7 9.158982 

8 5.891053 

9 3.60145* 

10 2.24833* 

TI 2 .00319* 

12 2.028723 

t3 3.221017 
FACTOR 

A(MID) 2.77088% 

A( CON) 2.41541* 

B 3.719875 

C 3.91692% 
OVERALL 4.07910* 
* SIGNIFICANT AT ALPHA = .05 
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APPENDIX C-4 


NAVAL HOSPITAL PENSACOLA 


ANCILLARY VS THE REST OF THE 


GENERAL 

ANCILLARV 
NON-ANCILLARY 
ANCILLARY STDEV 
NON-ANCILLARY STDEV 


GROUP FACTORS 


ANCILLARY 
NON-ANCILLARY 
ANCILLARY STDEV 
NON-ANCILLARY STDEV 


QUESTIONS 


ANCILLARY 
NON-ANCILLARY 
ANCILLARY STDEV 
NON-ANCILLARV STDEV 


QUESTIONS 


ANCILLARV 
NON-ANCILLARY 
ANCILLARY STDEV 
NON-ANCILLARY STDEV 


QUESTIONS 


ANCILLARY 
NON-ANCILLARY 
ANCILLARY STDEV 
NON-ANCILLARY STDEV 


AVERAGE 
0.72624 
0.70174 
1.75436 
1.80618 


A 
(MID) 


1.41176 
1.49057 
1.52979 
1.50625 


il 


ID 

. 49528 
133872 
. 50627 


HeH 


6 


. 39706 
. 59434 
97312 
39877 


PPPE 


11 
1.33471 
1.38208 

1.4714 
1.444 
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SURVEY GROUP 


INDEX 

11.49265 
97 311321 
21.45832 
14.53046 


A 
(CON) 


0.003676 
0.087264 
1.694494 
1.740047 


2 


-0.14706 
0.028302 
12800855 
1.883181 


7 


0.941176 
0-627338 
1.670326 
1.837196 


12 
0.154412 
0.146226 
1.566556 
1,581835 


0.70882 
0.69245 
1.73813 
1.76784 


3 


02173971 
-0.0566 
1.85968 
1.80842 


8 


0.91176 
0.89623 
1:67363 
1.70697 


13 
.72794 
.79245 
.85293 
1.2398 


PPOO 


Pilo. PP 


ROO 


73/6838 
SAA 
. 788388 
.897845 


4 


.69117 
1600377 
73330 
2331.26 


261029 
.66981 
273702 
. 76561 


SS ©) 


tà EOLO 


182393 
. 46698 
.67104 
.88185 


10 


82393 
.47642 
.68265 
273053 


ANCILLARY vs THE REST OF 


CALCULATED 


FACTOR 
A(MID) 
A(CON) 
B 
C 


OVERAL 


T-VALUE 
0.00000 
0.62229 
-0.69054 
-0.30014 
-1.25072 
0.78195 
-1.12362 
0.22088 
0.21962 
-1.27920 
0.22167 
0.23056 
0721223 


0.34149 
0.31046 
-0.03660 
-0.69144 


L 


APPENDIX C-5 


-1.50880 


* SIGNIFICANT AT ALPHA = 


THE SURVEY 
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GROUP 


APPENDIX C-6 


NAVAL HOSPITAL PENSACOLA 


ADMINISTRATIVE VS THE REST OF THE SURVEY GROUP 


GENERAL 

ADMIN SAT 
NON-ADMIN SAT 
ADMIN STDEV 
NON-ADMIN STDEV 


GROUP FACTORS 


ADMIN 

NON-ADMIN 

ADMIN STDEV 
NON-ADMIN STDEV 


QUESTIONS 


ADMIN SAT 
NON-ADMIN SAT 
ADMIN STDEV 
NON-ADMIN STDEV 


QUESTIONS 


ADMIN SAT 
NON-ADMIN SAT 
ADMIN STDEV 
NON-ADMIN STDEV 


QUESTIONS 


ADMIN SAT 
NON-ADMIN SAT 
ADMIN STDEV 
NON-ADMIN STDEV 


AVERAGE 
0.98764 
0.45342 
1.74163 
1.78861 


A 
(MID) 


1.7123 
1.22407 
1.44426 
1.54314 


l 


1.72024 
1.28889 
1.40121 
1299335 


6 


1.88095 
1017778 
1.41762 
1.262329 


kl 
1.593571 
1.203595 
1.49161 
1.40118 
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INDEX 

12.83929 
7.666667 
13,53834 
20.36609 


A 
(CON) 


0.309524 
-0.18333 

12 70929 
1.643083 


Z 


0.309524 
-0.36667 
1.883362 
1.763519 


7 


0.952381 
0.501111 

1.76544 
1: 773772 


12 
0.309524 
0.096939 
1.647337 
1.490712 


0.98148 
0.41444 
1.69909 

1.769 


3 


0.19048 
so a 589 
1281578 

io el 


8 


0357 
232718 
-522608 
. 75888 


PROP 


13 
1.10714 
0.45 
.71143 
. 86272 


Pr 


00993753 
020379 
1.828224 
1.863385 


4 


93833 
. 33888 
. 80394 
-87131 


HP POO 


297023 
.34444 
72318 
272987 


ROTO 


HP POO 


are 


-31067 
31067 
JS US 
.88702 


10 


. 68452 
- 18889 
27.0133 
.68914 


APPENDIX C-7 


ADMINISTRATIVE vs THE REST OF THE SURVEY GROUP 


CALCULATED T-VALUE 


il -1.88791 

2 -2.46426 * 
3 AMES LIZ 

4 = 2002/252360 
3 -2.21289 * 
6 =3.0221322 
7 -1.45634 

8 =3:0787573 
9 -2.40038 * 
10 -1.90959 
11 -1.50105 
12 -0.92410 
¡ES -2.43150 * 
FACTOR 


A(MID) -2.16394 * 

A(CON) -1.89548 * 

B -2.16356 * 

C -0.07198 * 

OVERALL 21.45925 

* SIGNIFICANT AT ALPHA = .05 


150 


APPENDIX C-8 


NAVAL HOSPITAL PENSACOLA 


ANCILLARY VS ADMINISTRATIVE GROUP 


GENERAL 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 


GROUP FACTORS 


ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 


QUESTIONS 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 


QUESTIONS 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 


QUESTIONS 
ANCILLARY SAT 
ADMIN SAT 
ANCILLARY STDEV 
ADMIN STDEV 


PPP Rh pp PHrOOo 


PRR PR 


PRR 


AVERAGE 
. 72624 
.98764 
. 75436 
2/4163 


A 


(MID) 


.41176 
>71230 
. 52979 
.44426 


. 50000 
.72024 
338/72 
.40121 


.39706 
.88095 
-9/312 
.41762 


11 


. 33471 
Doo L 
.47140 
.49161 


0. 


0. 
1% 
dui 


He OO 


HFPOO 


PIO 
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INDEX 
13.735530 
13281550 
25213973 
15.34150 

(CON) 
003676 O 
30954740 
694494 1 
709297 z1 

<147 10770, 
30052770 
.80086 1 
;BB34O4 1 
7 
.94118 0 
9523871 
267053 
.76544 1 
12 
„15441 0 
-30952 N1 
23060530F- 1 
.64734 1 


70882 
98148 
73813 
69909 


13974 


. 19048 
- 95968 
>81. 78 


.91176 
30357 
043603 
- 94208 


13 


.72794 
. 10714 
85293 
. 71143 


B 


C 


0.37684 
0.093765 
1.78839 
1.82822 


PROD 


SSI 


4 


“CITITE 
. 95833 
DIO 
. 80394 


61029 
. 97024 
“73102 
„72318 


PpOO 


HP pPOOo 


-82393 
-91667 
.67104 
07033 


10 


82353 
.68452 
- 69265 
27/0133 


APPENDIX C-9 


ANCILLARV vs ADMINISTRATIVE GROUP 


CALCULATED T-VALUE 


il 0.84816 
2 ISS 
3 1.66100 
4 0.93099 
9 0.36708 
6 2.05345 * 
7 0.03501 
8 1.38279 
9 1.27973 
10 -0.50710 
al 0.81736 
T2 0.56945 
13 1.31714 
FACTOR 


A(MID) 0.03318 
A(CON) 1.08134 
B 0.97679 
C 0.68113 
OVERALL 0.030183 


* SIGNIFICANT AT ALPHA = .05 
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APPENDIX C-10 


NAVAL HOSPITAL PENSACOLA 


SATISFACTION BASED ON TIME ON THE SYSTEM 


OVERALL 
TIME OF 
# PEOPLE USE 
35 121025 
34 6 TO 11 
104 >= 12 
174 
ADMINISTRATIVE 
TIME OF 
# PEOPLE USE 
19 1 TO 5 
17 6 TO 1l 
48 >= 12 
84 
ANCILLARY 
TIME OF 
# PEOPLE USE 
12 1-70=5 
13 6 TO 11 
43 >= 12 
68 
PHYSICIANS 
TIME OF 
# PEOPLE USE 
4 12170975 
4 6 TO 11 
14 >= 12 
22 


AVG SAT 
11.94 


12.66 
72938 


AVG SAT 
15.16 


16.07 
10.99 


AVG SAT 
2,36 


13.71 
9.45 


AVG SAT 
10.07 


3.54 
6.36 
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APPENDIX D-1 


SILAS B. HAYS ARMY HOSPITAL 


COMBINED RESULTS FOR ADMINISTRATIVE AND ANCILLARY 


TOTAL SATISFACTION AVERAGE INDEX 
ADMIN 0.79860 10.38182 
STAN DEV 1.76524 15.802753 
ANCILLARY 1.00000 13.00000 
STAN 1.42774 13.50000 
GROUP FACTORS A A B e 
(MID) (CON) 
ADMIN AVG 1.27878 0.05909 0.94545 0.34545 
ANCILLARY AVG 1.54545 0.37500 0.90000 0.87500 
ADMIN STDEV 1.74456 1.76356 1.67894 1.77449 
ANCILLARY STDEV 1.83031 2.34187 1.17898 1.72753 
COMBINED GROUP FACTORS A A B E 
(MID) (CON) 
AVERAGE 1.28739 0.07017 0.94386 0.36403 
STAN DEV 1.73276 1.78796 1.66396 1.77522 
INDIVIDUAL QUESTION RESPONSE 
QUESTION 1 2 3 4 
ADMIN 1.22727 -0.31818 0.118182 0.90909 
ANCILLARY 1.50000 O 0.750000 1.50000 
STAN DEV 1 2 3 4 
ADMIN 1.72499 1.73169 1.73588 1.56405 
ANCILLARY 1.50000 3.00000 0.829156 0.50000 
QUESTION 6 7 8 9 
ADMIN 1.15454 0.95454 1.20909 0.91818 
ANCILLARY 1.66666 1.25000 1.50000 O 
STAN DEV 6 7 8 9 
ADMIN 1.82501 1.691618 1.65746 1.70630 
ANCILLARY 0.47140 0.829156 0.50000 1.41421 
QUESTION 11 12 13 
ADMIN 1.45454 0.43636 0.84545 
ANCILLARY 1.50000 0.75000 0.75000 
STAN DEV 11 I2 13 
ADMIN 1.66589 1471372 1272250 
ANCILLARY 1.50000 1. 29903 1.29903 
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9 
0267272 
1.25000 


5 
1.79448 


028295 


10 


0.80000 


1.00000 


10 
1.58286 
1.00000 


ANCILLARV vs ADMINISTRATIVE 


CALCULATED T-VALUE 


l -0.22114 

2 -0.24072 

3 70; 309560 

4 -0.52982 

5 -0.45345 

6 -0.40038 

7 -0.24837 

8 -0.25320 

9 0.75420 

10 -0.17685 

LL -0.04192 

12 -0.25435 

13 0.07299 

FACTOR 

A(MID) -0.16331 
A(CON) -0.24566 
B 0.04106 
C -0.40956 
OVERALL 


* SIGNIFICANT AT ALPHA 


-0.23099 


APPENDIX D-2 


= .05 
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APPENDIX D-3 


SILAS B. HAYS ARMY HOSPITAL 


SATISFACTION BASED ON TIME OF USE 


OVERALL 
TIME OF 
# PEOPLE USE 
12 1 TO 5 
ll 6 TO 11 
34 >= 12 
57 
ADMINISTRATIVE 
TIME OF 
# PEOPLE USE 
12 1 TO 5 
9 6 TO 11 
34 >= 12 
SD 
ANCILLARY 
TIME OF 
# PEOPLE USE 
0 1. 7075 
2 6 TOTII 
0 >= 12 
2 


AVG SAT 


5.291667 
11. 22727 
12.05882 


AVG SAT 


5.291667 
10.83333 
12.05882 


AVG SAT 


N/A 
13 
N/A 
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APPENDIX E-l 


NAVAL HOSPITAL PENSACOLA AND SILAS B. HAYS ARMY HOSPITAL FT. ORD 


COMBINED RESULTS 


OVERALL 
AVERAGE 
STAND DEV 


TOTAL SATISFACTION 
PHYSICIAN 

STAND DEV 

ADMIN 

STAND DEV 
ANCILLARY 

STAND DEV 


GROUP FACTORS 


PHYSICIAN SAT 
ANCILLARY SAT 
ADMIN SAT 
PHYSICIAN STDEV 
ANCILLARY STDEV 
ADMIN STDEV 
OVERALL AVG 
OVERALL STDEV 


INDIVIDUAL QUESTION 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


STAN DEV 

PHYSICIAN 
ANCILLARY 
ADMIN 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


STAN DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


QUESTION 
0.7345987 
1.7789492 


QUESTION 
-0.38986 
1.6226442 
0.912839 
1.753449 
0.7340659 
1.7464702 


A 
(MID) 
.0606061 
.4119048 
. 5407674 
. 4361806 
1.524202 
1.5842124 
1.4163059 
1025737273 


eero 


RESPONSES 


1 
O. 6363636 
79 
1.9251799 


1 
1.582445 
1.5376234 
1.5562866 


6 
OLS 
123926571 
1.5935252 


6 
. 5883096 
.5569954 
.6304966 


Pre 


INDEX 
9.636364 
14.83246 


INDEX 

-4.15909 
9.406703 
11.86691 
14.52653 
9.464789 
14.38562 


A 
(CON) 
-0.76136 
0.252381 
0.210432 
1.538883 
1.717497 
1.771263 
0.058442 
1% 739173 


2 
-1.04545 
-0.14286 
0.061151 


2 
1.445283 
1.846121 
1.850491 


7 
-0.61364 
0795 
02953237 


7 


B 


-0.49545 
0.714286 
0.980576 
12538883 
1.724967 
1.691382 
0.759307 
1.717359 


3 

-1 
0.157143 
0.161871 


3 
. 430194 
.841084 
.784945 


Rp 


8 
70.65909 
0.928571 
1.266187 


8 


C 


IS 
0.391071 
0.495504 
1.595408 
1.810273 

18210123 
0.320346 
1.845252 


4 
-0.75 
0.714286 
0.938849 


4 
1.798042 
. 737345 
»21.32 106 


HE 


9 
-0.47727 
0.592857 

0.94964 


9 


5 
-1.25 
0.835714 
0.820144 


5 
1.263936 
1.654478 
1.724774 


10 
-0.31818 
0.371429 
O.730216 


10 


1.555283 1.460747 1.422045 1.606418 
1.653244 1.654616 1.731624 1.670513 
1.736606 1.684424 1.716713 1.656431 
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QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


STAN DEV 

PHYSICIAN 
ANCILLARY 
ADMIN 


cio 


RR 


Juli 12 


7954545 =0247727 
.3428571 0.171429 
- 903597 Tare 59712 


11 T2 


.0570015 1.097283 
.4724684 1.562703 
. 59634039 1.67507 
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T3 
-0.40909 
0.728571 
13003597 


13 
1.614116 
1.839421 
1.839421 


AQCESS FT ORD 


OVERALL 
PENSACOLA 
FT ORD 


PENSACOLA STDEV 
FT ORD STDEV 


GROUP FACTORS 


PENSACOLA 
FT ORD 


PENSACOLA STDEV 
FT ORD STDEV 


INDIVIDUAL QUESTION 


QUESTION 
PENSACOLA 
FT ORD 


STAN DEV 
PENSACOLA 
FT ORD 


QUESTION 
PENSACOLA 
FT ORD 


STAN DEV 
PENSACOLA 
FT ORD 


QUESTION 
PENSACOLA 
FT ORD 


STAN DEV 
PENSACOLA 
FT ORD 


APPENDIX E-2 


vs AQCESS PENSACOLA 


QUESTION INDEX 
02.7 1732 9.02586 
0.80567 10.47368 
1.78615 14.93465 
1.75489 15.73504 
A A B 
(MID) (CON) 
0.89719 0.05452 0.69885 
1.28363 -0.03955 0.94386 
1.51598 1.72164 1.76039 
1.13162. 1.77987 1.65140 
RESPONSES 
1 2 3 
1.49713 -0.04011 0.02012 
1.23684 -0.30702 0.14035 
1 2 3 
151903 128570072158 3113 
1.71834 1.79242 1.71616 
6 7 8 
1.51724 0.75000 0.90210 
1:15789 0.96491 1.21930 
6 7 8 
122607395 1.78052 1.69403 
1279981 1.66980 1.63170 
11 12 13 
1.36494 0.14943 0.76724 
1.45614 0.44444 0.84211 
11 12 13 
1.45492 1457589 223,382 113 
1.66038 1.65018 1.70951 
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C 


0.30582 
0.36403 
1.86665 
177558 


4 


2063733 
PASEO 


å 


, 86497 
.54306 


9 


.64655 
.88596 


9 


. 73474 
. 70530 


5 
0.60632 
09650239 


b 
1.581078 
1777273 


LO 
0.42816 
0.80702 


10 
1.71304 
1.56652 


APPENDIX E-3 


AQCESS FT ORD vs AQCESS PENSACOLA 


CALCULATED T-VALUE 


il -1.65958 
2 0.92434 
3 -0.43705 
4 71.06333 
5 -0.32762 
6 -1.34927 
7 -0.82097 
8 -1.17281 
9 -0.86718 
10 -0.47880 
TI -0.26139 
172 -1.17853 
13 0.37824 
FACTOR 

A(MID) -1.84777 
A(CON) 0232150 
B -0.75588 
E -0.17464 
OVERALL 


* SIGNIFICANT AT ALPHA 


-0.11841 


= .05 
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APPENDIX F-l 


AQCESS SILAS B. HAYS ARMY HOSPITAL FT ORD, CA VS AQCESS NAVAL HOSPITAL 
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PENSACOLA, FL VS CHCS NAVAL HOSPITAL CHARLESTON, SC 
TOTAL SATISFACTION QUESTION INDEX 
PENSACOLA 0471132 9.025386 
FT ORD 0.805367 10.47368 
CHARLESTON 0.86558 11.25248 
PENSACOLA STDEV 1278610 14.93465 
FT ORD STDEV 1.75489 15273594 
CHARLESTON STDEV 1.65718 110273883 
GROUP FACTORS A A B C 

(MID) (CON) 
PENSACOLA 0.89719 0.05452 0.69885 0.30582 
FT ORD 1.28363 -0.03955 0.94386 0.36403 
CHARLESTON 1.77288 -0.35891 0.77607 0.52847 
PENSACOLA STDEV 1515298 1.72164 71.760393: 1.866693 
FT ORD STDEV 121310602 1.77987: P:65140:- 1.77555 
CHARLESTON STDEV 1.17591 1.75196 1.51405 1.80255 
INDIVIDUAL QUESTION RESPONSES 
QUESTION 1 2 3 4 3 
PENSACOLA 1.49713 -1.04011 0.02012 0.63793 0.60632 
FT ORD 1.23684 -0.30702 0.14035 0.92998 0.69299 
CHARLESTON 1.67822 -0.50000 0.57426 0.97525 1.06436 
STAN DEV 1 2 3 4 5 
PENSACOLA Peo loos 1,857206791:831137 712864975 7 1.284078 
FT ORD 1.71834 1.279242 71027165 106251254306:. 271377273 
CHARLESTON 1.26586 1.75133 1.74690 1.67640 1.60559 
QUESTION 6 7 8 9 10 
PENSACOLA 1.51724 0.75000 0.90210 0.64655 0.42816 
FT ORD 12157289 0.96491 1.21930 0.88596 0.80702 
CHARLESTON 1.98515 0.79208 0.94060 1.16832 0.54455 
STAN DEV 6 7 8 9 10 
PENSACOLA 1250735 1.78052 1.69403 1.75474 1.71304 
ET ORD 1279951 15669807 22631702 71270530. 717750652 
CHARLESTON 1.04371 1.49837 1.43046 1.55541 1.18394 
QUESTION ll 12 13 
PENSACOLA 1.36494 0.14943 0.76724 
FT ORD 1.45614 0.44444 0.84211 
CHARLESTON 1.65347 -0.21287 0.59406 


STAN DEV ll 12 T3 


PENSACOLA 1.45492 1.57589 714782113 
FT ORD 1.66038 1.65018 1.70951 
CHARLESTON 1.18394 1.74471 1.48745 
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CHCS VS AQCESS 


TOTAL SATISFACTION 


CHCS 
AQCESS 


CHCS STDEV 


AQCESS 


GROUP FACTORS 


CHCS 
AQCESS 


CHCS- STDEV 
AQCESS STDEV 


APPENDIX F-2 


QUESTION 
0.86558 
0.73459 
1.65718 
ii 77895 


A 
(MID) 
1277288 
0.89898 
1217591 
100907377 


INDIVIDUAL QUESTION RESPONSES 


QUESTION 
CHCS 
AOCESS 


STAN DEV 
EHCS 
AQCESS 


QUESTION 
CHCS 
AQCESS 


STAN DEV 
CHCS 
AQCESS 


QUESTION 
CHCS 
AQCESS 


STAN DEV 
EHCS 
AQCESS 


1 
- 07822 
1.43290 


pa 


1 
1.26586 
1.57457 


6 
1.98515 
1.42857 


6 
1.04371 
1: 63907 


11 
1.65347 
1.38745 


17 
1.18394 
1.50874 


Ker OO 


PW 
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INDEX 
11.25248 
10713208 
112723883 
16.68588 
A B 
(CON) 
-338911,.0.77007 
.05844 0.75931 
-7919641151405 
«73917 1.74030 
Z 3 
. 50000 0.57426 
.10606 0.04978 
2 3 
279133 221274690 
.84215 1.80419 
7 8 
. 79208 0.94060 
.80303 X07 98052 
7 8 
.49837 1.43046 
.75630 1.68442 
12 13 
.21287 0.59406 
122294037785371 
12 13 
. 74471 1.48745 
.61301 1.79452 


C 


0.52847 
0.32035 
12.802955 
1.84525 


4 
0797525 
0.70996 


1.67640 
2271322 


1.120832 
0.705063 


1.55541 
1.74572 


.06436 
s6 2771 


“00299 
. 80185 


10 


.54455 
52165 


10 


.18394 
.68600 


APPENDIX F-3 


PENSACOLA vs CHARLESTON 


CALCULATED T-VALUE 


l 1.0038 
2 2.37 12982 
3 2.44113 * 
á 1.51899 
5 2.06814 * 
6 2.68497 * 
7 0.237516 
8 1.99893 
9 2.47011 * 
10 0.57204 
11 1.70795 
12 -1.76089 
13 -0.843187 
FACTOR 
A(MID) -5.044247 * 
A(CON) -1.89340 
B 0.382317 
C 0.953397 
OVERALL 1.28452 
* SIGNIFICANT AT ALPHA = .05 
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APPENDIX F-4 


FORT ORD vs CHARLESTON 


CALCULATED T-VALUE 


l -1.83458 
2 0.648655 
3 - 12.429233 
4 -0.148027 
5 41.33802 
6 -3.63284 * 
7 0.656472 
8 1.12314 
3 -1.03273 
10 1.22248 
11 =0. 88007 
12 2.29686 * 
jus: 0.95965 
FACTOR 


A(MID) -2.55907 * 
A( CON) 1209599 


B 0.61843 

C -0.572368 
OVERALL 20739393729 
| 
| * SIGNIFICANT AT ALPHA = .05 
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APPENDIX G-1 


COMBINED RESULTS FOR CHCS AND AQCESS 


GENERAL 


PHYSICIANS 


STDEV 
ANCILLARY 
STDEV 


ADMINISTRATIVE 


STDEV 


GROUP FACTORS 


PHYSICIAN 
ANCILLARY 
ADMIN SAT 
PHYSICIAN 
ANCILLARY 


SAT 
SAT 


STDEV 
STDEV 


ADMIN STDEV 
OVERALL AVG 
OVERALL STDEV 


INDIVIDUAL QUESTION RESPONSES 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


STAN DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


STAN DEV 
PHYSICIAN 
ANCILLARY 
ADMIN 


QUESTION 
PHYSICIAN 
ANCILLARY 
ADMIN 


AVERAGE 


RO Oro 


HeH Rh 


Rh pp 


- 10376 
-61997 
80593 
.73685 
.88229 
-73962 


A 


(MID) 
Ox 
1. 
I. 
un 
ika 
Ir 
ie: 
ie 


56202 
64286 
329993 
37496 
37705 
55679 
55063 
45332 


1 
1 


-03571 
333505 


il 


. 48637 
.41275 
233282 


6 


-02326 
73971 
. 63423 


6 


.46253 
. 36073 
-59816 


11 


SMI 
255714 
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INDEX 
1.348837 
11.15824 
11.42857 
13.86241 
11.4698 
14.25219 


A 
(CON) 
-0.46899 
0708571 
0.139262 
1.360055 
1.79179 
1.768214 
=0. 06777 
ili 291590 


2 
-0.72093 
-0.29286 
-0.01007 


2 
1.413688 
1.882234 
1.834576 


7 
-0.75581 
0.85 
0.922819 


7 
1.630128 
1.666583 
1.700005 


12 
-0.36145 
0.121429 
0.288591 
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B 


0.17907 
0.793571 
02927517 
1.505966 
1.683394 
1.673756 
0.763441 

1.66848 


3 
-0.27692 
0.467857 
0.167785 


3 
1.476515 
1.864816 
1.770006 


8 
-0.05814 
1.017857 
1.218121 


8 
1.630439 
1:5957/139 

1.306873 


13 
0.295181 
0.717857 
0.932886 


-0.48547 
07359821 
0.46896 
, 640625 
.857416 
.796915 
. 412634 
2837235 


Orr 


4 
0706153 
1.02857 
0.88590 


4 
1. 7/5381 
1769639 
1.70876 


9 
0.4418 
0.92143 
0.89261 


9 
1.63971 
71575 
1.70919 


3 
-0.092 
1.0397 
0.8322 


5 
1.8437 
1.6493 
1.6945 


10 
.2409 
, 4607 
.6711 


O O O 


10 
.4775 
.6664 
.6423 


Ra pp 


STAN DEV Jedi 12 13 


PHYSICIAN 1.23476 1.517594 1.485951 
ANCILLARY 1.35104 1.671047 1.749194 
ADMIN 1.53538 1.686087 1.749194 
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APPENDIX G-2 


COMBINED RESULTS FOR CHCS AND AQCESS 
PHYSICIANS VS THE REST OF THE SURVEY GROUP 


GENERAL 
PHYSICIANS 
STDEV 
NON-PHYSICIANS 
STDEV 


GROUP FACTORS 


PHYSICIAN SAT 


PHYSICIAN STDEV 


NON-PHYSICIANS 


NON-PHYSICIANS STDEV 


QUESTION 
PHYSICIAN 
NON-PHYSICIANS 


STAN DEV 
PHYSICIAN 
NON-PHYSICIANS 


QUESTION 
PHYSICIAN 
NON-PHYSICIANS 


STAN DEV 
PHYSICIAN 
NON-PHYSICIANS 


QUESTION 
PHYSICIAN 
NON-PHYSICIANS 


STAN DEV 
PHYSICIAN 
NON-PHYSICIANS 


AVERAGE 
0.10376 
1.48637 
0.87437 
1793957 


A 
(MID) 
0.56202 
1.37496 
1.59804 

1.4731 


1 
l 
1.58304 


il 
1.48637 
1.47676 


6 
1.02326 
1.68339 


6 
1.46253 
1.48875 


101 
137952 
1.52768 


11 
1.23476 
1.44929 


INDEX 
1.348837 
11.15824 
11.44983 
14.06473 
A 
(CON) 
-0.46899 
1.360055 
0.295848 
1.783479 
2 
-0.72093 
-0.27682 
2 
1.413688 
1.863184 
7 
-0.75581 
-0.14706 
7 
1.630128 
1.68429 
12 
-0.36145 
0.205882 
152 
1.517594 
1.681623 
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B 


0.17907 
1.505966 
0.86263 
1.679766 


3 
-0.27692 
0.313149 


$ 
1.476515 
1.822732 


8 
-0.05814 
1er 2107 


8 
1.630439 
1.562716 


13 
0.295181 
0.82872 


13 
1.485951 
1.730074 


E 


-0.48547 
1.640625 
0.512976 
1.827038 


4 
0.061538 
0.955017 


4 
1.753811 
1.704274 


3 
0.44186 
0.906574 


9 
1.639705 
1.712439 


5 
-0.092 3 
0.930796 


9 
1.843711 
1.675816 


10 
0.240964 
0.569204 


10 
1.477464 
1.65739 


PHYSICIANS vs THE REST OF THE SURVEY 


CALCULATED T-VALUE 


Prem 
ONDHODSDOSAUN$PWNDH 


FACTOR 
A(MID) 
A(CON) 
B 
C 


OVERALL 


* SIGNIFICANT AT ALPHA 


274072 UR 
1.52642 
1.983087 
=- 2319355 
35651435 
2: 7/8690% 
22203232 
4.62484* 
1.68446 
1.19979 
0.64683 
2.108637 
1.91295 


42383097 
22092282 
2.49046* 
3.41617* 


4.52104* 


APPENDIX G-3 


05 


GROUP 
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APPENDIX G-4 


COMBINED RESULTS FOR CHCS AND AQCESS 


ANCILLARY VS THE REST OF THE 


GENERAL 

ANCILLARY 

ANCILLARY STDEV 
NON-ANCILLARY 
NON-ANCILLARY STDEV 


GROUP FACTORS 


ANCILLARY SAT 
ANCILLARY STDEV 
NON-ANCILLARY 
NON-ANCILLARY STDEV 


QUESTION 
ANCILLARY 
NON-ANCILLARY 


STAN DEV 
ANCILLARY 
NON-ANCILLARY 


QUESTION 
ANCILLARY 
NON-ANCILLARY 


STAN DEV 
ANCILLARY 
NON-ANCILLARY 


QUESTION 
ANCILLARY 
NON-ANCILLARY 


STAN DEV 
ANCILLARY 
NON-ANCILLARY 


AVERAGE 
0.86621 
1.73663 
0.70753 
1.74605 


A 
(MID) 


1.64286 
1.37705 
1 . 43837 
1.53366 


1 
. 63571 
.41406 


1 
.41275 
938/1 


6 
273071 
, 4974 


6 
. 36073 
. 38935 


Dl 
.55714 
.40365 


ll 
. 35104 
. 4688 


SURVEY GROUP 


INDEX 

11.42857 
13.86241 
92 2031725 
14.25906 


A 
(CON) 
-0.08571 
1.79179 
-0.05599 
1.72398 


2 
-0.29286 
-0.17708 


2 
1.882234 
1.770553 


7 
0.85 
0.763021 


7 
1.666583 
1.746181 


12 
0.121429 
0.065104 


12 


1.671047 
1.660958 


170 


HP Or © 


B 


793571 
.683394 
2727309 
-667132 


3 


. 467897 
. 020833 


3 


.864816 
.730421 


8 


017857 
. 932292 


8 


-5957139 
. 647472 


13 


. 71857 
.734375 


13 


.749194 
-677754 


D 


0.560714 
1.856906 
0.254557 
1.807729 


4 
1.028571 


0.617188 


4 
1.696395 
1.790256 


2 


0.921429 
0.791667 


9 
1.715758 
1.690455 


5 
1.039286 
0.557292 


5 
1.64595 
1.80331 


10 
0.460714 
0.578125 


10 
1.666442 
1.608616 


APPENDIX G-5 


ANCILLARY vs THE REST OF THE SURVEY GROUP 


CALCULATED T-VALUE 
l -1.45036 
2 0.61534 
3 -2.40663* 
4 -2.18425* 
5 -2.55840* 
6 -1.44183 
Y -0.49104 
8 -0.46473 
3 70. 7/1419 
10 0.68605 
11 -1.06003 
12 -0.30922 
13 0.10936 
FACTOR 


A(MID) -1.28482 
A(CON) 0.11483 


B -0.33643 
C -1.56279 
OVERALL -1.48159 
* SIGNIFICANT AT ALPHA = .05 
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APPENDIX G-6 


COMBINED RESULTS FOR CHCS AND AQCESS 


ADMINISTRATIVE VS THE REST OF THE SURVEX GROUP 


GENERAL 
ADMINISTRATIVE 
ADMIN STDEV 
NON-ADMIN 
NON-ADMIN STDEV 


GROUP FACTORS 


ADMIN SAT 

ADMIN STDEV 
NON-ADMIN SAT 
NON-ADMIN STDEV 


QUESTION 
ADMIN 
NON-ADMIN 


STAN DEV 
ADMIN 
NON-ADMIN 


QUESTION 
ADMIN 
NON-ADMIN 


STAN 
ADMIN 
NON-ADMIN 


DEV 


QUESTION 
ADMIN 
NON-ADMIN 


STAN DEV 
ADMIN 
NON-ADMIN 


AVERAGE 


0.88178 


ie 


74002 


0.68705 


I: 


74022 


A 


(MID) 


INDEX 
11.4698 
14.2522 


9.06011 
14.25906 


A 
(CON) 


B 


1.55593 0.137584 0.927517 
1.55679 1.768819 


Je 
il 


le 
. 48634 


1 


52302 
40079 


1 
93356 


1 


„33282 
. 45556 


6 


. 63423 
190831 


6 


-59816 
.41787 


VI 
I 


. 44262 


11 


„339 38 
32266 


-0.23634 
1.721812 


2 
-0.01007 
-0.40164 


2 
1.834576 
1.794009 


7 
0.922819 
0.699454 


7 
1.700005 
1.658675 


12 
02283235 
-0.07104 


12 


1.687653 
1.629772 


172 


1.673796 
0.64918 
1: 66395 


3 
1.672785 
.243169 


OO 


3 
. 770006 
.826758 


Hp 


8 
1.218121 
0.765027 


8 
1.960173 
1.621064 


13 
0.932886 
0.560109 


13 
1.749194 
1.692298 


z 


0.468121 
1.797367 
0.314891 
1.862015 


4 
0.885906 
0.713115 


4 
1.708762 
1.802304 


9 
0.892617 
0.808743 


9 
1. 709T97 
1.695874 


5 
. 828859 
. 704918 


OO 


9 
1.697088 
1.798703 


10 
0.671141 
0.412568 


10 
1.642322 
1.618458 


APPENDIX G-7 


ADMINISTRATIVE vs THE REST OF THE SURVEY GROUP 


CALCULATED T-VALUE 
0.00000 
22.01622 = 
-0.36259 
-0.95500 
-69139 
-0.43885 
-1.09948 
-2.65698 
-0.52758 
10 -1.57692 
11 -0.39612 
12 -1.97450 * 
E3 -2.00788 * 


OONJNOOkWbbDHM 
t 
O 


FACTOR 
A(MID) -0.25394 
A(CON) -1.82145 


B -1.08437 
C -1.01902 
OVERALL -1.59613 
* SIGNIFICANT AT ALPHA = .05 
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